2024-09-15T20:45:05.075 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T20:45:05.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T20:45:05.187 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637 branch: main description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-bitmap} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905637' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: global: mon election default strategy: 1 osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd pool default ec fast read: true osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 2 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - mon.a - - osd.4 - osd.5 - osd.6 - osd.7 - mon.b - mgr.x - - osd.8 - osd.9 - osd.10 - osd.11 - mon.c seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwTUze3CAEVj9L1G7fXAlFONBA1gNoENTTzotqZuZ1kv4imoACnlmnDdd0R8aSoWyzrNAQUR1aH6Ld/dHHHQEA= smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFzrKEwKx2okjgEV5HtFqiO9twfbAKevzS8YKqVEmcj8VQLqhY4tt8x5jFtFAEjl6cgdfWpUMFSNwlaPZ6LaAls= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8/s6YWJ82ZJBnguSXXgjgiHXYVUOdP4LhsEpZUSj02TV/fl3m8t8N/jgYJ2+smazf6EQSptSNGuAbh2dkxJzU= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 4 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 4 l: 3 m: 2 name: lrcprofile plugin: lrc objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T20:45:05.187 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T20:45:05.188 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T20:45:05.188 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T20:45:05.189 INFO:teuthology.task.internal:Checking packages... 2024-09-15T20:45:05.217 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T20:45:05.217 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T20:45:05.217 INFO:teuthology.packaging:ref: None 2024-09-15T20:45:05.217 INFO:teuthology.packaging:tag: None 2024-09-15T20:45:05.217 INFO:teuthology.packaging:branch: main 2024-09-15T20:45:05.217 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:45:05.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-15T20:45:05.470 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:45:05.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T20:45:05.478 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T20:45:05.479 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T20:45:05.524 INFO:teuthology.task.internal:Saving configuration 2024-09-15T20:45:05.542 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T20:45:05.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T20:45:05.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:38:14.495804', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINmyPZ61Hc6WaceHScdtpqpvvx0JZ+6NW/V9xgp8Eetv'} 2024-09-15T20:45:05.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:38:14.499367', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDc2s5jNgpOoILnQZAIIOSvXjKHOdKkJifIF30WD4LFz'} 2024-09-15T20:45:05.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:38:14.497859', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINlbJ+FQuBHW07jP1NMIiiobP0ulvduEKXG1lRZEEhOD'} 2024-09-15T20:45:05.666 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T20:45:05.675 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2024-09-15T20:45:05.676 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2024-09-15T20:45:05.676 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2024-09-15T20:45:05.676 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T20:45:05.795 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa1b603b130>, signals=[15]) 2024-09-15T20:45:05.795 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T20:45:05.801 INFO:teuthology.task.internal:Opening connections... 2024-09-15T20:45:05.801 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:45:05.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:45:05.892 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:45:05.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:45:05.974 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:45:05.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:45:06.060 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T20:45:06.067 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-15T20:45:06.073 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-15T20:45:06.074 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="Ubuntu" 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="22.04" 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_CODENAME=jammy 2024-09-15T20:45:06.122 INFO:teuthology.orchestra.run.smithi027.stdout:ID=ubuntu 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE=debian 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T20:45:06.123 INFO:teuthology.orchestra.run.smithi027.stdout:UBUNTU_CODENAME=jammy 2024-09-15T20:45:06.124 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-15T20:45:06.148 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-09-15T20:45:06.153 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-09-15T20:45:06.153 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Ubuntu" 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="22.04" 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_CODENAME=jammy 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:ID=ubuntu 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE=debian 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T20:45:06.199 INFO:teuthology.orchestra.run.smithi066.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T20:45:06.200 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T20:45:06.200 INFO:teuthology.orchestra.run.smithi066.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T20:45:06.200 INFO:teuthology.orchestra.run.smithi066.stdout:UBUNTU_CODENAME=jammy 2024-09-15T20:45:06.200 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-09-15T20:45:06.232 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-15T20:45:06.237 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-15T20:45:06.237 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="22.04" 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=jammy 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2024-09-15T20:45:06.284 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2024-09-15T20:45:06.285 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T20:45:06.285 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T20:45:06.285 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T20:45:06.285 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T20:45:06.285 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=jammy 2024-09-15T20:45:06.285 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-15T20:45:06.307 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T20:45:06.316 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T20:45:06.376 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T20:45:06.376 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-15T20:45:06.379 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-09-15T20:45:06.382 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-15T20:45:06.386 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T20:45:06.398 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T20:45:06.399 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-15T20:45:06.427 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-09-15T20:45:06.430 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-15T20:45:06.456 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T20:45:06.536 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T20:45:06.554 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T20:45:06.554 INFO:teuthology.task.kernel:config {'osd.0': {'kdb': 1, 'sha1': 'distro'}, 'osd.4': {'kdb': 1, 'sha1': 'distro'}, 'osd.8': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T20:45:06.555 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T20:45:06.555 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T20:45:06.556 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T20:45:06.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:45:06.561 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-15T20:45:06.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:45:06.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:45:06.563 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-15T20:45:06.563 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-15T20:45:06.606 INFO:teuthology.orchestra.run.smithi066.stdout:5.15.0-56-generic 2024-09-15T20:45:06.607 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.15.0-56-generic 2024-09-15T20:45:06.607 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get clean 2024-09-15T20:45:06.608 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-56-generic 2024-09-15T20:45:06.609 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.15.0-56-generic 2024-09-15T20:45:06.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-09-15T20:45:06.610 INFO:teuthology.orchestra.run.smithi027.stdout:5.15.0-56-generic 2024-09-15T20:45:06.610 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.15.0-56-generic 2024-09-15T20:45:06.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-get clean 2024-09-15T20:45:06.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-get update 2024-09-15T20:45:06.743 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get update 2024-09-15T20:45:06.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-15T20:45:06.866 INFO:teuthology.orchestra.run.smithi027.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T20:45:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:45:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T20:45:06.881 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T20:45:06.919 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:45:06.938 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T20:45:06.952 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T20:45:06.982 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:45:07.002 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T20:45:07.037 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T20:45:07.064 INFO:teuthology.orchestra.run.smithi027.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T20:45:07.087 INFO:teuthology.orchestra.run.smithi027.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T20:45:07.109 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T20:45:07.175 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T20:45:07.190 INFO:teuthology.orchestra.run.smithi027.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T20:45:07.194 INFO:teuthology.orchestra.run.smithi027.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T20:45:07.196 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T20:45:07.197 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T20:45:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T20:45:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T20:45:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T20:45:07.217 INFO:teuthology.orchestra.run.smithi027.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T20:45:07.221 INFO:teuthology.orchestra.run.smithi027.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T20:45:07.221 INFO:teuthology.orchestra.run.smithi027.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T20:45:07.226 INFO:teuthology.orchestra.run.smithi027.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T20:45:07.234 INFO:teuthology.orchestra.run.smithi027.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T20:45:07.235 INFO:teuthology.orchestra.run.smithi027.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T20:45:07.236 INFO:teuthology.orchestra.run.smithi027.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T20:45:07.236 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T20:45:07.236 INFO:teuthology.orchestra.run.smithi027.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T20:45:07.236 INFO:teuthology.orchestra.run.smithi027.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T20:45:07.237 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T20:45:07.238 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T20:45:07.238 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T20:45:07.243 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T20:45:07.248 INFO:teuthology.orchestra.run.smithi027.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T20:45:07.257 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T20:45:07.257 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T20:45:07.260 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T20:45:07.260 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T20:45:07.264 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T20:45:07.270 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T20:45:07.279 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T20:45:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T20:45:07.282 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T20:45:07.283 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T20:45:07.299 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T20:45:07.303 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T20:45:07.303 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T20:45:07.308 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T20:45:07.309 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T20:45:07.309 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T20:45:07.309 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T20:45:07.309 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T20:45:07.310 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T20:45:07.310 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T20:45:07.312 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T20:45:07.313 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T20:45:07.316 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T20:45:07.318 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T20:45:07.319 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T20:45:07.319 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T20:45:07.338 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T20:45:07.339 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T20:45:07.342 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T20:45:07.342 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T20:45:07.342 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T20:45:07.343 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T20:45:07.343 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T20:45:07.344 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T20:45:07.344 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T20:45:07.350 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T20:45:07.350 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T20:45:07.350 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T20:45:07.350 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T20:45:07.354 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T20:45:07.356 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T20:45:07.356 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T20:45:07.356 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T20:45:07.362 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T20:45:07.362 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T20:45:07.363 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T20:45:07.363 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T20:45:07.368 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T20:45:07.368 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T20:45:07.374 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T20:45:07.374 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T20:45:07.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T20:45:07.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T20:45:07.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T20:45:07.379 INFO:teuthology.orchestra.run.smithi066.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T20:45:07.419 INFO:teuthology.orchestra.run.smithi027.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T20:45:07.493 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T20:45:07.557 INFO:teuthology.orchestra.run.smithi027.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T20:45:07.752 INFO:teuthology.orchestra.run.smithi027.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T20:45:07.779 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T20:45:07.912 INFO:teuthology.orchestra.run.smithi027.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T20:45:07.926 INFO:teuthology.orchestra.run.smithi027.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T20:45:07.926 INFO:teuthology.orchestra.run.smithi027.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T20:45:07.928 INFO:teuthology.orchestra.run.smithi027.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T20:45:07.952 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T20:45:07.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T20:45:07.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T20:45:07.967 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T20:45:07.997 INFO:teuthology.orchestra.run.smithi027.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T20:45:08.013 INFO:teuthology.orchestra.run.smithi027.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T20:45:08.013 INFO:teuthology.orchestra.run.smithi027.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T20:45:08.035 INFO:teuthology.orchestra.run.smithi027.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T20:45:08.051 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T20:45:08.059 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T20:45:08.060 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T20:45:08.063 INFO:teuthology.orchestra.run.smithi027.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T20:45:08.069 INFO:teuthology.orchestra.run.smithi027.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T20:45:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T20:45:08.078 INFO:teuthology.orchestra.run.smithi027.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T20:45:08.078 INFO:teuthology.orchestra.run.smithi027.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T20:45:08.089 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T20:45:08.110 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T20:45:08.115 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T20:45:08.116 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T20:45:08.117 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T20:45:08.117 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T20:45:08.157 INFO:teuthology.orchestra.run.smithi027.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T20:45:08.157 INFO:teuthology.orchestra.run.smithi027.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T20:45:08.158 INFO:teuthology.orchestra.run.smithi027.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T20:45:08.159 INFO:teuthology.orchestra.run.smithi027.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T20:45:08.159 INFO:teuthology.orchestra.run.smithi027.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T20:45:08.160 INFO:teuthology.orchestra.run.smithi027.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T20:45:08.161 INFO:teuthology.orchestra.run.smithi027.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T20:45:08.197 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T20:45:08.241 INFO:teuthology.orchestra.run.smithi027.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T20:45:08.241 INFO:teuthology.orchestra.run.smithi027.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T20:45:13.628 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 16.0 MB in 2s (7,427 kB/s) 2024-09-15T20:45:13.870 INFO:teuthology.orchestra.run.smithi027.stdout:Fetched 16.0 MB in 2s (7,250 kB/s) 2024-09-15T20:45:14.021 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 16.0 MB in 2s (7,180 kB/s) 2024-09-15T20:45:15.159 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:45:15.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:45:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:45:15.308 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:45:15.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:45:15.378 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:45:15.449 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-15T20:45:15.450 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-15T20:45:15.458 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:45:15.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:45:15.540 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:45:15.568 INFO:teuthology.orchestra.run.smithi027.stdout:Building dependency tree... 2024-09-15T20:45:15.568 INFO:teuthology.orchestra.run.smithi027.stdout:Reading state information... 2024-09-15T20:45:15.621 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:45:15.621 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:45:15.621 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:45:15.622 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:45:15.622 INFO:teuthology.orchestra.run.smithi092.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T20:45:15.622 INFO:teuthology.orchestra.run.smithi092.stdout: python2.7-minimal 2024-09-15T20:45:15.622 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:45:15.623 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-09-15T20:45:15.623 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:15.623 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T20:45:15.624 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:15.625 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-09-15T20:45:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T20:45:15.625 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-09-15T20:45:15.625 INFO:teuthology.orchestra.run.smithi092.stdout: thermald 2024-09-15T20:45:15.676 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-09-15T20:45:15.676 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:15.676 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T20:45:15.676 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:15.677 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2024-09-15T20:45:15.678 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T20:45:15.739 INFO:teuthology.orchestra.run.smithi027.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:45:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:45:15.739 INFO:teuthology.orchestra.run.smithi027.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:45:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:45:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T20:45:15.740 INFO:teuthology.orchestra.run.smithi027.stdout: python2.7-minimal 2024-09-15T20:45:15.740 INFO:teuthology.orchestra.run.smithi027.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:45:15.741 INFO:teuthology.orchestra.run.smithi027.stdout:The following additional packages will be installed: 2024-09-15T20:45:15.741 INFO:teuthology.orchestra.run.smithi027.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T20:45:15.742 INFO:teuthology.orchestra.run.smithi027.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:15.743 INFO:teuthology.orchestra.run.smithi027.stdout:Suggested packages: 2024-09-15T20:45:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T20:45:15.743 INFO:teuthology.orchestra.run.smithi027.stdout:Recommended packages: 2024-09-15T20:45:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: thermald 2024-09-15T20:45:15.755 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-09-15T20:45:15.755 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-09-15T20:45:15.797 INFO:teuthology.orchestra.run.smithi027.stdout:The following NEW packages will be installed: 2024-09-15T20:45:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T20:45:15.798 INFO:teuthology.orchestra.run.smithi027.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:15.799 INFO:teuthology.orchestra.run.smithi027.stdout:The following packages will be upgraded: 2024-09-15T20:45:15.799 INFO:teuthology.orchestra.run.smithi027.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T20:45:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:45:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 113 MB of archives. 2024-09-15T20:45:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T20:45:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T20:45:15.952 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:45:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:45:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:45:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:45:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T20:45:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: python2.7-minimal 2024-09-15T20:45:15.953 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:45:15.955 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-09-15T20:45:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T20:45:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:15.957 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-09-15T20:45:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T20:45:15.957 INFO:teuthology.orchestra.run.smithi066.stdout:Recommended packages: 2024-09-15T20:45:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: thermald 2024-09-15T20:45:15.978 INFO:teuthology.orchestra.run.smithi027.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:45:15.978 INFO:teuthology.orchestra.run.smithi027.stdout:Need to get 113 MB of archives. 2024-09-15T20:45:15.979 INFO:teuthology.orchestra.run.smithi027.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T20:45:15.979 INFO:teuthology.orchestra.run.smithi027.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T20:45:16.016 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-09-15T20:45:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T20:45:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T20:45:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T20:45:16.017 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages will be upgraded: 2024-09-15T20:45:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T20:45:16.071 INFO:teuthology.orchestra.run.smithi066.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:45:16.071 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 113 MB of archives. 2024-09-15T20:45:16.071 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T20:45:16.071 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T20:45:16.496 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T20:45:16.594 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T20:45:17.241 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T20:45:17.361 INFO:teuthology.orchestra.run.smithi027.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T20:45:17.432 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T20:45:17.432 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T20:45:17.432 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T20:45:17.563 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T20:45:17.585 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T20:45:17.595 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T20:45:17.728 INFO:teuthology.orchestra.run.smithi027.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T20:45:18.055 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 113 MB in 2s (69.7 MB/s) 2024-09-15T20:45:18.216 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T20:45:19.835 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T20:45:19.835 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T20:45:19.836 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T20:45:19.916 INFO:teuthology.orchestra.run.smithi027.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T20:45:19.917 INFO:teuthology.orchestra.run.smithi027.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T20:45:19.917 INFO:teuthology.orchestra.run.smithi027.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T20:45:20.142 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T20:45:20.147 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:20.174 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T20:45:20.212 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:20.254 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T20:45:20.434 INFO:teuthology.orchestra.run.smithi027.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T20:45:20.525 INFO:teuthology.orchestra.run.smithi027.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T20:45:20.664 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-15T20:45:20.887 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T20:45:20.931 INFO:teuthology.orchestra.run.smithi027.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-09-15T20:45:21.104 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T20:45:22.363 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T20:45:22.385 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:22.520 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:22.963 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T20:45:22.967 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:22.991 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T20:45:23.012 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:23.049 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:23.129 INFO:teuthology.orchestra.run.smithi027.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T20:45:23.133 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:23.218 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:23.317 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:25.578 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T20:45:25.599 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:25.627 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T20:45:25.647 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:25.677 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:25.753 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:26.172 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T20:45:26.193 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:26.213 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T20:45:26.230 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:26.233 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:26.263 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:29.389 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:29.515 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:29.840 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:29.943 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:30.322 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T20:45:30.344 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T20:45:30.388 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:32.615 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:32.731 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:33.281 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:33.418 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:33.704 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T20:45:33.716 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T20:45:33.725 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:33.754 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:33.875 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:34.242 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:34.387 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:34.698 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T20:45:34.720 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T20:45:34.789 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:36.445 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T20:45:36.482 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:36.519 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:39.124 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:39.247 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:39.581 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:39.630 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T20:45:39.667 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:39.702 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:39.704 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:39.803 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T20:45:39.914 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:41.278 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T20:45:41.290 INFO:teuthology.orchestra.run.smithi066.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:45:41.290 INFO:teuthology.orchestra.run.smithi066.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T20:45:41.290 INFO:teuthology.orchestra.run.smithi066.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:45:41.290 INFO:teuthology.orchestra.run.smithi066.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:41.315 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T20:45:41.353 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:41.459 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:43.310 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:43.351 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:43.475 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:43.808 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:43.930 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:44.056 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T20:45:44.192 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:44.711 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T20:45:44.812 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T20:45:44.914 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:44.981 INFO:teuthology.orchestra.run.smithi066.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T20:45:44.981 INFO:teuthology.orchestra.run.smithi066.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:45.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T20:45:45.348 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T20:45:45.537 INFO:teuthology.orchestra.run.smithi027.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:45:45.537 INFO:teuthology.orchestra.run.smithi027.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T20:45:45.537 INFO:teuthology.orchestra.run.smithi027.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:45:45.537 INFO:teuthology.orchestra.run.smithi027.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:45.739 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T20:45:45.895 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:46.021 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T20:45:46.182 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:47.288 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:47.539 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:45:47.539 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T20:45:47.539 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:45:47.539 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:47.729 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:49.331 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T20:45:49.457 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T20:45:49.555 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:49.592 INFO:teuthology.orchestra.run.smithi027.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:49.653 INFO:teuthology.orchestra.run.smithi027.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T20:45:49.654 INFO:teuthology.orchestra.run.smithi027.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:50.989 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T20:45:51.115 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T20:45:51.275 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T20:45:51.348 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T20:45:51.348 INFO:teuthology.orchestra.run.smithi092.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:45:59.652 INFO:teuthology.orchestra.run.smithi066.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T20:45:59.652 INFO:teuthology.orchestra.run.smithi066.stdout:Sourcing file `/etc/default/grub' 2024-09-15T20:45:59.681 INFO:teuthology.orchestra.run.smithi066.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:45:59.707 INFO:teuthology.orchestra.run.smithi066.stdout:Generating grub configuration file ... 2024-09-15T20:45:59.946 INFO:teuthology.orchestra.run.smithi066.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:45:59.960 INFO:teuthology.orchestra.run.smithi066.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:00.228 INFO:teuthology.orchestra.run.smithi066.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:00.233 INFO:teuthology.orchestra.run.smithi066.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:00.302 INFO:teuthology.orchestra.run.smithi066.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:00.309 INFO:teuthology.orchestra.run.smithi066.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:00.481 INFO:teuthology.orchestra.run.smithi066.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:00.482 INFO:teuthology.orchestra.run.smithi066.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:00.482 INFO:teuthology.orchestra.run.smithi066.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:00.520 INFO:teuthology.orchestra.run.smithi066.stdout:done 2024-09-15T20:46:00.986 DEBUG:teuthology.orchestra.run.smithi066:> dpkg -s linux-image-generic 2024-09-15T20:46:01.010 INFO:teuthology.orchestra.run.smithi066.stdout:Package: linux-image-generic 2024-09-15T20:46:01.010 INFO:teuthology.orchestra.run.smithi066.stdout:Status: install ok installed 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Priority: optional 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Section: kernel 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Installed-Size: 21 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: amd64 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Source: linux-meta 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Recommends: thermald 2024-09-15T20:46:01.011 INFO:teuthology.orchestra.run.smithi066.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:01.012 INFO:teuthology.orchestra.run.smithi066.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:01.012 INFO:teuthology.orchestra.run.smithi066.stdout: available. 2024-09-15T20:46:01.012 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T20:46:01.012 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T20:46:01.012 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T20:46:01.012 INFO:teuthology.task.kernel:Installing distro kernel on osd.4... 2024-09-15T20:46:01.012 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T20:46:01.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get clean 2024-09-15T20:46:01.208 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get update 2024-09-15T20:46:01.322 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:46:01.330 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:46:01.347 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:46:01.443 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:46:02.814 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:46:02.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:46:02.889 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:46:03.080 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-09-15T20:46:03.081 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-09-15T20:46:03.258 INFO:teuthology.orchestra.run.smithi066.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T20:46:03.258 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:46:03.258 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:46:03.258 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:46:03.258 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:46:03.259 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:46:03.259 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:46:03.259 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:46:03.298 INFO:teuthology.orchestra.run.smithi066.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:46:03.300 DEBUG:teuthology.orchestra.run.smithi066:> dpkg -s linux-image-generic 2024-09-15T20:46:03.354 INFO:teuthology.orchestra.run.smithi066.stdout:Package: linux-image-generic 2024-09-15T20:46:03.354 INFO:teuthology.orchestra.run.smithi066.stdout:Status: install ok installed 2024-09-15T20:46:03.354 INFO:teuthology.orchestra.run.smithi066.stdout:Priority: optional 2024-09-15T20:46:03.354 INFO:teuthology.orchestra.run.smithi066.stdout:Section: kernel 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Installed-Size: 21 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: amd64 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Source: linux-meta 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Recommends: thermald 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:03.355 INFO:teuthology.orchestra.run.smithi066.stdout: available. 2024-09-15T20:46:03.356 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-15T20:46:03.406 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.QQ2uNufL0K 2024-09-15T20:46:03.407 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub/grub.cfg /tmp/tmp.QQ2uNufL0K 2024-09-15T20:46:03.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.QQ2uNufL0K 2024-09-15T20:46:03.596 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.QQ2uNufL0K is 10KB 2024-09-15T20:46:03.651 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.QQ2uNufL0K 2024-09-15T20:46:03.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:03.716 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:46:03.716 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T20:46:03.716 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:03.782 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T20:46:03.782 DEBUG:teuthology.orchestra.run.smithi066:> sudo update-grub 2024-09-15T20:46:04.022 INFO:teuthology.orchestra.run.smithi027.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T20:46:04.023 INFO:teuthology.orchestra.run.smithi027.stdout:Sourcing file `/etc/default/grub' 2024-09-15T20:46:04.028 INFO:teuthology.orchestra.run.smithi027.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:46:04.040 INFO:teuthology.orchestra.run.smithi027.stdout:Generating grub configuration file ... 2024-09-15T20:46:04.334 INFO:teuthology.orchestra.run.smithi027.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:46:04.350 INFO:teuthology.orchestra.run.smithi027.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:04.666 INFO:teuthology.orchestra.run.smithi027.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:04.672 INFO:teuthology.orchestra.run.smithi027.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:04.908 INFO:teuthology.orchestra.run.smithi027.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:04.909 INFO:teuthology.orchestra.run.smithi027.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:04.909 INFO:teuthology.orchestra.run.smithi027.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:04.941 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-09-15T20:46:04.990 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub' 2024-09-15T20:46:04.991 INFO:teuthology.orchestra.run.smithi066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:46:04.994 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-09-15T20:46:05.183 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:46:05.189 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:05.411 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:05.431 DEBUG:teuthology.orchestra.run.smithi027:> dpkg -s linux-image-generic 2024-09-15T20:46:05.459 INFO:teuthology.orchestra.run.smithi066.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:05.464 INFO:teuthology.orchestra.run.smithi066.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Package: linux-image-generic 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Status: install ok installed 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Priority: optional 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Section: kernel 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Installed-Size: 21 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: amd64 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Source: linux-meta 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:05.503 INFO:teuthology.orchestra.run.smithi027.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:05.504 INFO:teuthology.orchestra.run.smithi027.stdout:Recommends: thermald 2024-09-15T20:46:05.504 INFO:teuthology.orchestra.run.smithi027.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:05.504 INFO:teuthology.orchestra.run.smithi027.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:05.504 INFO:teuthology.orchestra.run.smithi027.stdout: available. 2024-09-15T20:46:05.504 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T20:46:05.504 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T20:46:05.504 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T20:46:05.504 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2024-09-15T20:46:05.504 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T20:46:05.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-get clean 2024-09-15T20:46:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:05.611 INFO:teuthology.orchestra.run.smithi066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:05.621 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-09-15T20:46:05.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2024-09-15T20:46:05.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-get update 2024-09-15T20:46:05.857 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T20:46:05.857 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub' 2024-09-15T20:46:05.887 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:46:05.913 INFO:teuthology.orchestra.run.smithi092.stdout:Generating grub configuration file ... 2024-09-15T20:46:05.931 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:46:05.932 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:46:06.002 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:46:06.083 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:46:06.235 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:46:06.252 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:06.629 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:06.634 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:06.704 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:06.709 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:06.902 INFO:teuthology.orchestra.run.smithi092.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:06.902 INFO:teuthology.orchestra.run.smithi092.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:06.903 INFO:teuthology.orchestra.run.smithi092.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:06.947 INFO:teuthology.orchestra.run.smithi092.stdout:done 2024-09-15T20:46:07.385 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:46:07.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:46:07.468 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:46:07.493 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:07.514 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-09-15T20:46:07.515 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:07.515 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-09-15T20:46:07.515 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T20:46:07.515 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T20:46:07.515 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T20:46:07.515 INFO:teuthology.task.kernel:Installing distro kernel on osd.8... 2024-09-15T20:46:07.515 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T20:46:07.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-09-15T20:46:07.685 INFO:teuthology.orchestra.run.smithi027.stdout:Building dependency tree... 2024-09-15T20:46:07.685 INFO:teuthology.orchestra.run.smithi027.stdout:Reading state information... 2024-09-15T20:46:07.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-15T20:46:07.799 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:46:07.885 INFO:teuthology.orchestra.run.smithi027.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T20:46:07.886 INFO:teuthology.orchestra.run.smithi027.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:46:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:46:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:46:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:46:07.886 INFO:teuthology.orchestra.run.smithi027.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:46:07.887 INFO:teuthology.orchestra.run.smithi027.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:46:07.887 INFO:teuthology.orchestra.run.smithi027.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:46:07.921 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:46:07.931 INFO:teuthology.orchestra.run.smithi027.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:46:07.933 DEBUG:teuthology.orchestra.run.smithi027:> dpkg -s linux-image-generic 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Package: linux-image-generic 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Status: install ok installed 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Priority: optional 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Section: kernel 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Installed-Size: 21 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:07.950 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: amd64 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Source: linux-meta 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Recommends: thermald 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:07.951 INFO:teuthology.orchestra.run.smithi027.stdout: available. 2024-09-15T20:46:07.951 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-15T20:46:07.997 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.G9ABra4ebe 2024-09-15T20:46:07.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub/grub.cfg /tmp/tmp.G9ABra4ebe 2024-09-15T20:46:08.013 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:46:08.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.G9ABra4ebe 2024-09-15T20:46:08.085 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:46:08.183 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.G9ABra4ebe is 10KB 2024-09-15T20:46:08.196 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.G9ABra4ebe 2024-09-15T20:46:08.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:08.257 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:46:08.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T20:46:08.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:08.329 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T20:46:08.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo update-grub 2024-09-15T20:46:09.248 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:46:09.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T20:46:09.331 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:46:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:Sourcing file `/etc/default/grub' 2024-09-15T20:46:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:46:09.545 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-15T20:46:09.547 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-15T20:46:09.549 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-15T20:46:09.741 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T20:46:09.741 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:46:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:46:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:46:09.741 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:46:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:46:09.742 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:46:09.742 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:46:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:46:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:09.784 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T20:46:09.785 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-09-15T20:46:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.121.121 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-09-15T20:46:09.803 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-09-15T20:46:09.803 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-15T20:46:09.846 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.Wu2ZPzqgns 2024-09-15T20:46:09.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Wu2ZPzqgns 2024-09-15T20:46:09.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.Wu2ZPzqgns 2024-09-15T20:46:09.974 INFO:teuthology.orchestra.run.smithi027.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:09.980 INFO:teuthology.orchestra.run.smithi027.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:10.028 INFO:teuthology.orchestra.run.smithi027.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:10.036 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.Wu2ZPzqgns is 10KB 2024-09-15T20:46:10.049 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.Wu2ZPzqgns 2024-09-15T20:46:10.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:10.110 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:46:10.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T20:46:10.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T20:46:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:10.183 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T20:46:10.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo update-grub 2024-09-15T20:46:10.184 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-15T20:46:10.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-15T20:46:11.682 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub' 2024-09-15T20:46:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T20:46:11.690 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-09-15T20:46:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T20:46:11.957 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T20:46:12.233 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T20:46:12.238 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T20:46:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T20:46:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T20:46:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T20:46:12.428 INFO:teuthology.orchestra.run.smithi092.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T20:46:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T20:46:12.440 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-09-15T20:46:12.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-09-15T20:46:35.649 INFO:teuthology.task.kernel:Checking client osd.4 for new kernel version... 2024-09-15T20:46:35.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:46:35.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:46:40.201 INFO:teuthology.task.kernel:Checking client osd.0 for new kernel version... 2024-09-15T20:46:40.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:46:40.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:46:42.447 INFO:teuthology.task.kernel:Checking client osd.8 for new kernel version... 2024-09-15T20:46:42.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:46:42.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:46:54.013 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-15T20:46:58.621 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T20:47:00.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T20:47:03.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:47:03.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:06.077 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-15T20:47:07.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:47:07.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:09.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:47:09.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:10.688 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T20:47:12.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T20:47:18.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:47:18.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:21.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-15T20:47:22.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:47:22.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:24.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:47:24.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:25.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T20:47:36.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:47:36.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:40.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:47:40.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:43.425 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-09-15T20:47:43.837 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T20:47:58.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:47:58.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:47:58.860 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-15T20:48:00.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:48:00.013 INFO:teuthology.task.kernel:Checking kernel version of osd.8, want "5.15.0-121-generic"... 2024-09-15T20:48:00.013 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-09-15T20:48:00.018 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-121-generic 2024-09-15T20:48:00.019 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T20:48:00.019 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T20:48:00.019 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T20:48:01.020 INFO:teuthology.task.kernel:Enabling kdb on osd.8... 2024-09-15T20:48:01.020 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T20:48:01.362 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-09-15T20:48:01.393 DEBUG:teuthology.parallel:result is None 2024-09-15T20:48:01.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:48:01.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:48:20.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-15T20:48:36.161 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T20:48:41.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:48:41.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:48:41.533 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-15T20:48:42.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:48:42.128 INFO:teuthology.task.kernel:Checking kernel version of osd.0, want "5.15.0-121-generic"... 2024-09-15T20:48:42.128 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-15T20:48:42.177 INFO:teuthology.orchestra.run.smithi027.stdout:5.15.0-121-generic 2024-09-15T20:48:42.177 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T20:48:42.177 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T20:48:42.177 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T20:48:43.178 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2024-09-15T20:48:43.178 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T20:48:43.262 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-09-15T20:48:43.286 DEBUG:teuthology.parallel:result is None 2024-09-15T20:48:54.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:48:54.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:48:54.362 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-15T20:48:54.908 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:48:54.908 INFO:teuthology.task.kernel:Checking kernel version of osd.4, want "5.15.0-121-generic"... 2024-09-15T20:48:54.908 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-15T20:48:54.954 INFO:teuthology.orchestra.run.smithi066.stdout:5.15.0-121-generic 2024-09-15T20:48:54.955 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T20:48:54.955 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T20:48:54.955 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T20:48:55.956 INFO:teuthology.task.kernel:Enabling kdb on osd.4... 2024-09-15T20:48:55.956 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T20:48:56.057 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-09-15T20:48:56.080 DEBUG:teuthology.parallel:result is None 2024-09-15T20:48:56.080 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T20:48:56.086 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T20:48:56.087 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T20:48:56.089 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T20:48:56.092 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T20:48:56.098 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T20:48:56.130 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T20:48:56.215 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T20:48:56.216 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T20:48:56.218 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T20:48:56.221 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T20:48:56.238 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T20:48:56.262 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T20:48:56.262 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T20:48:56.274 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T20:48:56.276 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T20:48:56.290 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.299 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.299 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.306 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.307 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T20:48:56.309 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T20:48:56.315 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T20:48:56.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T20:48:56.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T20:48:56.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T20:48:56.373 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T20:48:56.382 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T20:48:56.382 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T20:48:56.414 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T20:48:56.417 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T20:48:56.423 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T20:48:56.465 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T20:48:56.513 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:48:56.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T20:48:56.577 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T20:48:56.583 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T20:48:56.631 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:48:56.631 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T20:48:56.687 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T20:48:56.693 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T20:48:56.739 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:48:56.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T20:48:56.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-15T20:48:56.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-09-15T20:48:56.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-15T20:48:56.892 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T20:48:56.900 INFO:teuthology.task.internal:Starting timer... 2024-09-15T20:48:56.900 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T20:48:56.935 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T20:48:56.981 DEBUG:teuthology.task.selinux:Excluding smithi027: OS 'ubuntu' does not support SELinux 2024-09-15T20:48:56.981 DEBUG:teuthology.task.selinux:Excluding smithi066: OS 'ubuntu' does not support SELinux 2024-09-15T20:48:56.982 DEBUG:teuthology.task.selinux:Excluding smithi092: OS 'ubuntu' does not support SELinux 2024-09-15T20:48:56.982 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T20:48:56.982 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T20:48:56.982 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T20:48:56.982 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T20:48:57.031 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T20:48:57.034 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T20:48:57.165 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T20:48:57.176 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T20:48:57.177 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi066.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T20:56:38.867 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-09-15T20:56:38.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:56:38.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:56:38.959 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-15T20:56:39.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-15T20:56:39.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:56:39.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:56:39.119 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-15T20:56:39.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-15T20:56:39.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:56:39.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T20:56:39.276 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-15T20:56:39.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-15T20:56:39.354 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T20:56:39.364 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T20:56:39.364 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T20:56:39.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T20:56:39.367 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T20:56:39.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T20:56:39.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T20:56:39.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T20:56:39.396 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Command line: ntpd -gq 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: ---------------------------------------------------- 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: corporation. Support and training for ntp-4 are 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: available at https://www.nwtime.org/support 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: ---------------------------------------------------- 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: proto: precision = 0.045 usec (-24) 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: basedate set to 2022-02-04 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: gps base set to 2022-02-06 (week 2196) 2024-09-15T20:56:39.397 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.398 INFO:teuthology.orchestra.run.smithi066.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.398 INFO:teuthology.orchestra.run.smithi066.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen normally on 3 enp3s0f1 172.21.15.66:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen normally on 4 lo [::1]:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8bd%5]:123 2024-09-15T20:56:39.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:39 ntpd[18546]: Listening on routing socket on fd #22 for interface updates 2024-09-15T20:56:39.400 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T20:56:39.400 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Command line: ntpd -gq 2024-09-15T20:56:39.400 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: ---------------------------------------------------- 2024-09-15T20:56:39.400 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T20:56:39.400 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: corporation. Support and training for ntp-4 are 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: available at https://www.nwtime.org/support 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: ---------------------------------------------------- 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: proto: precision = 0.046 usec (-24) 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: basedate set to 2022-02-04 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: gps base set to 2022-02-06 (week 2196) 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.402 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.402 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T20:56:39.402 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T20:56:39.402 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T20:56:39.403 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen normally on 3 ens1f0 172.21.15.27:123 2024-09-15T20:56:39.403 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen normally on 4 lo [::1]:123 2024-09-15T20:56:39.403 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:395a%4]:123 2024-09-15T20:56:39.403 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:39 ntpd[18356]: Listening on routing socket on fd #22 for interface updates 2024-09-15T20:56:39.421 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T20:56:39.421 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Command line: ntpd -gq 2024-09-15T20:56:39.421 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: ---------------------------------------------------- 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: corporation. Support and training for ntp-4 are 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: available at https://www.nwtime.org/support 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: ---------------------------------------------------- 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: proto: precision = 0.046 usec (-24) 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: basedate set to 2022-02-04 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: gps base set to 2022-02-06 (week 2196) 2024-09-15T20:56:39.422 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T20:56:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen normally on 3 enp3s0f1 172.21.15.92:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen normally on 4 lo [::1]:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cd45%5]:123 2024-09-15T20:56:39.424 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:39 ntpd[18534]: Listening on routing socket on fd #22 for interface updates 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:40 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:40 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:40 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:41.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:41 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:42 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:42 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:42.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:42 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:43.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:43.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:43 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:43 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:44 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:44 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:44 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:45 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:45 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:46.398 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.399 INFO:teuthology.orchestra.run.smithi066.stdout:15 Sep 20:56:46 ntpd[18546]: ntpd: time slew +0.001863 s 2024-09-15T20:56:46.399 INFO:teuthology.orchestra.run.smithi066.stdout:ntpd: time slew +0.001863s 2024-09-15T20:56:46.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T20:56:46.399 INFO:teuthology.orchestra.run.smithi066.stderr:15 Sep 20:56:46 ntpd[18546]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:46 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:46.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:46.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:46.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:46 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:46.455 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T20:56:46.456 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-09-15T20:56:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:46.456 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:47 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:47.420 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stdout:15 Sep 20:56:47 ntpd[18534]: ntpd: time slew -0.002029 s 2024-09-15T20:56:47.421 INFO:teuthology.orchestra.run.smithi092.stdout:ntpd: time slew -0.002029s 2024-09-15T20:56:47.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T20:56:47.422 INFO:teuthology.orchestra.run.smithi092.stderr:15 Sep 20:56:47 ntpd[18534]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T20:56:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T20:56:47.473 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-09-15T20:56:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:47.473 INFO:teuthology.orchestra.run.smithi092.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:47.474 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stdout:15 Sep 20:56:48 ntpd[18356]: ntpd: time slew +0.002072 s 2024-09-15T20:56:48.399 INFO:teuthology.orchestra.run.smithi027.stdout:ntpd: time slew +0.002072s 2024-09-15T20:56:48.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T20:56:48.400 INFO:teuthology.orchestra.run.smithi027.stderr:15 Sep 20:56:48 ntpd[18356]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T20:56:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T20:56:48.448 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================== 2024-09-15T20:56:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T20:56:48.449 INFO:teuthology.run_tasks:Running task install... 2024-09-15T20:56:48.457 DEBUG:teuthology.task.install:project ceph 2024-09-15T20:56:48.457 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T20:56:48.457 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T20:56:48.457 INFO:teuthology.task.install:Using flavor: default 2024-09-15T20:56:48.463 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T20:56:48.463 INFO:teuthology.task.install:extra packages: [] 2024-09-15T20:56:48.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-key list | grep Ceph 2024-09-15T20:56:48.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-key list | grep Ceph 2024-09-15T20:56:48.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-key list | grep Ceph 2024-09-15T20:56:48.528 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T20:56:48.532 INFO:teuthology.orchestra.run.smithi066.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T20:56:48.551 INFO:teuthology.orchestra.run.smithi027.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T20:56:48.559 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T20:56:48.559 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T20:56:48.560 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T20:56:48.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:56:48.565 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T20:56:48.565 INFO:teuthology.orchestra.run.smithi066.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T20:56:48.566 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T20:56:48.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:56:48.582 INFO:teuthology.orchestra.run.smithi027.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T20:56:48.582 INFO:teuthology.orchestra.run.smithi027.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T20:56:48.583 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T20:56:48.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:56:48.705 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T20:56:48.706 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:48.711 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T20:56:48.711 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:48.727 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T20:56:48.727 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:48.818 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:56:48.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T20:56:48.825 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:56:48.825 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T20:56:48.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-09-15T20:56:48.840 DEBUG:teuthology.orchestra.run.smithi066:> sudo apt-get update 2024-09-15T20:56:48.849 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:56:48.849 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T20:56:48.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo apt-get update 2024-09-15T20:56:48.998 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:56:49.017 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:56:49.026 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:56:49.044 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:56:49.054 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T20:56:49.067 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T20:56:49.076 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T20:56:49.088 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T20:56:49.090 INFO:teuthology.orchestra.run.smithi027.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T20:56:49.098 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T20:56:49.110 INFO:teuthology.orchestra.run.smithi066.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T20:56:49.111 INFO:teuthology.orchestra.run.smithi027.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T20:56:49.120 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T20:56:49.131 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T20:56:49.134 INFO:teuthology.orchestra.run.smithi027.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T20:56:49.137 INFO:teuthology.orchestra.run.smithi066.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:56:49.138 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:56:49.147 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T20:56:49.158 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T20:56:49.163 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T20:56:49.164 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T20:56:49.164 INFO:teuthology.orchestra.run.smithi027.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T20:56:49.192 INFO:teuthology.orchestra.run.smithi027.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T20:56:49.220 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:56:49.236 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T20:56:49.302 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:56:49.319 INFO:teuthology.orchestra.run.smithi027.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T20:56:49.376 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 29.2 kB in 0s (83.5 kB/s) 2024-09-15T20:56:49.450 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 29.2 kB in 0s (67.5 kB/s) 2024-09-15T20:56:49.569 INFO:teuthology.orchestra.run.smithi027.stdout:Fetched 29.2 kB in 1s (57.6 kB/s) 2024-09-15T20:56:50.419 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:56:50.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:50.461 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:56:50.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:50.500 INFO:teuthology.orchestra.run.smithi066.stdout:Reading package lists... 2024-09-15T20:56:50.540 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-09-15T20:56:50.616 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:56:50.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:56:50.697 INFO:teuthology.orchestra.run.smithi027.stdout:Reading package lists... 2024-09-15T20:56:50.714 INFO:teuthology.orchestra.run.smithi066.stdout:Building dependency tree... 2024-09-15T20:56:50.715 INFO:teuthology.orchestra.run.smithi066.stdout:Reading state information... 2024-09-15T20:56:50.748 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-09-15T20:56:50.749 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-09-15T20:56:50.906 INFO:teuthology.orchestra.run.smithi027.stdout:Building dependency tree... 2024-09-15T20:56:50.906 INFO:teuthology.orchestra.run.smithi027.stdout:Reading state information... 2024-09-15T20:56:50.933 INFO:teuthology.orchestra.run.smithi066.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:56:50.933 INFO:teuthology.orchestra.run.smithi066.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:56:50.933 INFO:teuthology.orchestra.run.smithi066.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:56:50.933 INFO:teuthology.orchestra.run.smithi066.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:56:50.934 INFO:teuthology.orchestra.run.smithi066.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:56:50.934 INFO:teuthology.orchestra.run.smithi066.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:56:50.934 INFO:teuthology.orchestra.run.smithi066.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:56:50.935 INFO:teuthology.orchestra.run.smithi066.stdout:The following additional packages will be installed: 2024-09-15T20:56:50.935 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T20:56:50.935 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T20:56:50.935 INFO:teuthology.orchestra.run.smithi066.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T20:56:50.935 INFO:teuthology.orchestra.run.smithi066.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T20:56:50.936 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T20:56:50.936 INFO:teuthology.orchestra.run.smithi066.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T20:56:50.936 INFO:teuthology.orchestra.run.smithi066.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T20:56:50.937 INFO:teuthology.orchestra.run.smithi066.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet zip 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout:Suggested packages: 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T20:56:50.938 INFO:teuthology.orchestra.run.smithi066.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T20:56:50.939 INFO:teuthology.orchestra.run.smithi066.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T20:56:50.939 INFO:teuthology.orchestra.run.smithi066.stdout:Recommended packages: 2024-09-15T20:56:50.939 INFO:teuthology.orchestra.run.smithi066.stdout: btrfs-tools 2024-09-15T20:56:50.942 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:56:50.942 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:56:50.942 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:56:50.942 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:56:50.942 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:56:50.943 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:56:50.943 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T20:56:50.944 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T20:56:50.945 INFO:teuthology.orchestra.run.smithi092.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T20:56:50.945 INFO:teuthology.orchestra.run.smithi092.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T20:56:50.945 INFO:teuthology.orchestra.run.smithi092.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T20:56:50.945 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T20:56:50.946 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet zip 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T20:56:50.947 INFO:teuthology.orchestra.run.smithi092.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T20:56:50.948 INFO:teuthology.orchestra.run.smithi092.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T20:56:50.948 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-09-15T20:56:50.948 INFO:teuthology.orchestra.run.smithi092.stdout: btrfs-tools 2024-09-15T20:56:51.047 INFO:teuthology.orchestra.run.smithi066.stdout:The following NEW packages will be installed: 2024-09-15T20:56:51.047 INFO:teuthology.orchestra.run.smithi066.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T20:56:51.048 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T20:56:51.048 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T20:56:51.048 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T20:56:51.048 INFO:teuthology.orchestra.run.smithi066.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T20:56:51.048 INFO:teuthology.orchestra.run.smithi066.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T20:56:51.050 INFO:teuthology.orchestra.run.smithi066.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T20:56:51.051 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi066.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T20:56:51.052 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T20:56:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T20:56:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T20:56:51.110 INFO:teuthology.orchestra.run.smithi066.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T20:56:51.110 INFO:teuthology.orchestra.run.smithi066.stdout:Need to get 244 MB of archives. 2024-09-15T20:56:51.110 INFO:teuthology.orchestra.run.smithi066.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T20:56:51.110 INFO:teuthology.orchestra.run.smithi066.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T20:56:51.112 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T20:56:51.112 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 244 MB of archives. 2024-09-15T20:56:51.112 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T20:56:51.112 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T20:56:51.125 INFO:teuthology.orchestra.run.smithi027.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T20:56:51.125 INFO:teuthology.orchestra.run.smithi027.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T20:56:51.125 INFO:teuthology.orchestra.run.smithi027.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T20:56:51.126 INFO:teuthology.orchestra.run.smithi027.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T20:56:51.126 INFO:teuthology.orchestra.run.smithi027.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T20:56:51.126 INFO:teuthology.orchestra.run.smithi027.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T20:56:51.126 INFO:teuthology.orchestra.run.smithi027.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T20:56:51.127 INFO:teuthology.orchestra.run.smithi027.stdout:The following additional packages will be installed: 2024-09-15T20:56:51.127 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T20:56:51.127 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T20:56:51.127 INFO:teuthology.orchestra.run.smithi027.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T20:56:51.128 INFO:teuthology.orchestra.run.smithi027.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T20:56:51.128 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T20:56:51.129 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet zip 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout:Suggested packages: 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T20:56:51.130 INFO:teuthology.orchestra.run.smithi027.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout:Recommended packages: 2024-09-15T20:56:51.131 INFO:teuthology.orchestra.run.smithi027.stdout: btrfs-tools 2024-09-15T20:56:51.195 INFO:teuthology.orchestra.run.smithi066.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T20:56:51.197 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T20:56:51.200 INFO:teuthology.orchestra.run.smithi066.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T20:56:51.201 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T20:56:51.204 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T20:56:51.206 INFO:teuthology.orchestra.run.smithi066.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T20:56:51.221 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T20:56:51.222 INFO:teuthology.orchestra.run.smithi066.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout:The following NEW packages will be installed: 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T20:56:51.241 INFO:teuthology.orchestra.run.smithi027.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T20:56:51.242 INFO:teuthology.orchestra.run.smithi027.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T20:56:51.243 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T20:56:51.244 INFO:teuthology.orchestra.run.smithi027.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T20:56:51.280 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T20:56:51.281 INFO:teuthology.orchestra.run.smithi066.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T20:56:51.285 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T20:56:51.286 INFO:teuthology.orchestra.run.smithi066.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T20:56:51.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T20:56:51.299 INFO:teuthology.orchestra.run.smithi066.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T20:56:51.301 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T20:56:51.303 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T20:56:51.303 INFO:teuthology.orchestra.run.smithi066.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T20:56:51.304 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T20:56:51.304 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T20:56:51.305 INFO:teuthology.orchestra.run.smithi066.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T20:56:51.305 INFO:teuthology.orchestra.run.smithi066.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T20:56:51.306 INFO:teuthology.orchestra.run.smithi066.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T20:56:51.312 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T20:56:51.315 INFO:teuthology.orchestra.run.smithi066.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T20:56:51.315 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T20:56:51.317 INFO:teuthology.orchestra.run.smithi066.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T20:56:51.317 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T20:56:51.319 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T20:56:51.319 INFO:teuthology.orchestra.run.smithi066.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T20:56:51.319 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T20:56:51.320 INFO:teuthology.orchestra.run.smithi066.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T20:56:51.321 INFO:teuthology.orchestra.run.smithi066.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T20:56:51.336 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T20:56:51.337 INFO:teuthology.orchestra.run.smithi066.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T20:56:51.338 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T20:56:51.339 INFO:teuthology.orchestra.run.smithi027.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T20:56:51.339 INFO:teuthology.orchestra.run.smithi027.stdout:Need to get 244 MB of archives. 2024-09-15T20:56:51.339 INFO:teuthology.orchestra.run.smithi027.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T20:56:51.339 INFO:teuthology.orchestra.run.smithi027.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T20:56:51.340 INFO:teuthology.orchestra.run.smithi066.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T20:56:51.340 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T20:56:51.341 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T20:56:51.341 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T20:56:51.341 INFO:teuthology.orchestra.run.smithi066.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T20:56:51.342 INFO:teuthology.orchestra.run.smithi066.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T20:56:51.342 INFO:teuthology.orchestra.run.smithi066.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T20:56:51.343 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T20:56:51.344 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T20:56:51.344 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T20:56:51.344 INFO:teuthology.orchestra.run.smithi066.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T20:56:51.353 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T20:56:51.354 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T20:56:51.354 INFO:teuthology.orchestra.run.smithi066.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T20:56:51.355 INFO:teuthology.orchestra.run.smithi066.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T20:56:51.355 INFO:teuthology.orchestra.run.smithi066.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T20:56:51.355 INFO:teuthology.orchestra.run.smithi066.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T20:56:51.370 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T20:56:51.371 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T20:56:51.371 INFO:teuthology.orchestra.run.smithi066.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T20:56:51.371 INFO:teuthology.orchestra.run.smithi066.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T20:56:51.373 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T20:56:51.374 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T20:56:51.374 INFO:teuthology.orchestra.run.smithi066.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T20:56:51.374 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T20:56:51.375 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T20:56:51.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T20:56:51.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T20:56:51.375 INFO:teuthology.orchestra.run.smithi066.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T20:56:51.376 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T20:56:51.376 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T20:56:51.387 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T20:56:51.388 INFO:teuthology.orchestra.run.smithi066.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T20:56:51.389 INFO:teuthology.orchestra.run.smithi066.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T20:56:51.389 INFO:teuthology.orchestra.run.smithi066.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T20:56:51.389 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T20:56:51.391 INFO:teuthology.orchestra.run.smithi066.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T20:56:51.398 INFO:teuthology.orchestra.run.smithi066.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T20:56:51.405 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T20:56:51.405 INFO:teuthology.orchestra.run.smithi066.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T20:56:51.406 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T20:56:51.406 INFO:teuthology.orchestra.run.smithi066.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T20:56:51.412 INFO:teuthology.orchestra.run.smithi066.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T20:56:51.412 INFO:teuthology.orchestra.run.smithi066.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T20:56:51.412 INFO:teuthology.orchestra.run.smithi066.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T20:56:51.413 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T20:56:51.413 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T20:56:51.413 INFO:teuthology.orchestra.run.smithi066.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T20:56:51.413 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T20:56:51.414 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T20:56:51.416 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T20:56:51.418 INFO:teuthology.orchestra.run.smithi092.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T20:56:51.422 INFO:teuthology.orchestra.run.smithi092.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T20:56:51.422 INFO:teuthology.orchestra.run.smithi066.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T20:56:51.424 INFO:teuthology.orchestra.run.smithi092.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T20:56:51.424 INFO:teuthology.orchestra.run.smithi066.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T20:56:51.425 INFO:teuthology.orchestra.run.smithi066.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T20:56:51.426 INFO:teuthology.orchestra.run.smithi066.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T20:56:51.427 INFO:teuthology.orchestra.run.smithi092.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T20:56:51.436 INFO:teuthology.orchestra.run.smithi027.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T20:56:51.438 INFO:teuthology.orchestra.run.smithi092.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T20:56:51.439 INFO:teuthology.orchestra.run.smithi027.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T20:56:51.439 INFO:teuthology.orchestra.run.smithi066.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T20:56:51.442 INFO:teuthology.orchestra.run.smithi066.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T20:56:51.442 INFO:teuthology.orchestra.run.smithi092.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T20:56:51.442 INFO:teuthology.orchestra.run.smithi066.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T20:56:51.443 INFO:teuthology.orchestra.run.smithi092.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T20:56:51.443 INFO:teuthology.orchestra.run.smithi066.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T20:56:51.444 INFO:teuthology.orchestra.run.smithi092.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T20:56:51.446 INFO:teuthology.orchestra.run.smithi066.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T20:56:51.448 INFO:teuthology.orchestra.run.smithi092.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T20:56:51.452 INFO:teuthology.orchestra.run.smithi027.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T20:56:51.455 INFO:teuthology.orchestra.run.smithi027.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T20:56:51.476 INFO:teuthology.orchestra.run.smithi066.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T20:56:51.490 INFO:teuthology.orchestra.run.smithi066.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T20:56:51.496 INFO:teuthology.orchestra.run.smithi066.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T20:56:51.497 INFO:teuthology.orchestra.run.smithi066.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T20:56:51.501 INFO:teuthology.orchestra.run.smithi066.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T20:56:51.501 INFO:teuthology.orchestra.run.smithi066.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T20:56:51.501 INFO:teuthology.orchestra.run.smithi066.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T20:56:51.501 INFO:teuthology.orchestra.run.smithi066.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T20:56:51.503 INFO:teuthology.orchestra.run.smithi066.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T20:56:51.503 INFO:teuthology.orchestra.run.smithi066.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T20:56:51.504 INFO:teuthology.orchestra.run.smithi066.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T20:56:51.508 INFO:teuthology.orchestra.run.smithi092.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T20:56:51.509 INFO:teuthology.orchestra.run.smithi092.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T20:56:51.510 INFO:teuthology.orchestra.run.smithi092.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T20:56:51.511 INFO:teuthology.orchestra.run.smithi092.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T20:56:51.512 INFO:teuthology.orchestra.run.smithi092.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T20:56:51.521 INFO:teuthology.orchestra.run.smithi066.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T20:56:51.522 INFO:teuthology.orchestra.run.smithi027.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T20:56:51.523 INFO:teuthology.orchestra.run.smithi066.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T20:56:51.523 INFO:teuthology.orchestra.run.smithi066.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T20:56:51.526 INFO:teuthology.orchestra.run.smithi066.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T20:56:51.528 INFO:teuthology.orchestra.run.smithi027.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T20:56:51.529 INFO:teuthology.orchestra.run.smithi092.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T20:56:51.531 INFO:teuthology.orchestra.run.smithi092.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T20:56:51.531 INFO:teuthology.orchestra.run.smithi092.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T20:56:51.538 INFO:teuthology.orchestra.run.smithi066.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T20:56:51.538 INFO:teuthology.orchestra.run.smithi066.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T20:56:51.539 INFO:teuthology.orchestra.run.smithi066.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T20:56:51.539 INFO:teuthology.orchestra.run.smithi066.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T20:56:51.539 INFO:teuthology.orchestra.run.smithi066.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T20:56:51.540 INFO:teuthology.orchestra.run.smithi066.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T20:56:51.541 INFO:teuthology.orchestra.run.smithi027.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T20:56:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T20:56:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T20:56:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T20:56:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T20:56:51.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T20:56:51.549 INFO:teuthology.orchestra.run.smithi092.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T20:56:51.549 INFO:teuthology.orchestra.run.smithi027.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T20:56:51.549 INFO:teuthology.orchestra.run.smithi027.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T20:56:51.550 INFO:teuthology.orchestra.run.smithi027.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T20:56:51.550 INFO:teuthology.orchestra.run.smithi027.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T20:56:51.555 INFO:teuthology.orchestra.run.smithi066.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T20:56:51.555 INFO:teuthology.orchestra.run.smithi066.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T20:56:51.557 INFO:teuthology.orchestra.run.smithi027.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T20:56:51.559 INFO:teuthology.orchestra.run.smithi066.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T20:56:51.559 INFO:teuthology.orchestra.run.smithi027.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T20:56:51.561 INFO:teuthology.orchestra.run.smithi027.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T20:56:51.562 INFO:teuthology.orchestra.run.smithi092.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T20:56:51.562 INFO:teuthology.orchestra.run.smithi092.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T20:56:51.563 INFO:teuthology.orchestra.run.smithi027.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T20:56:51.563 INFO:teuthology.orchestra.run.smithi027.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T20:56:51.565 INFO:teuthology.orchestra.run.smithi092.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T20:56:51.568 INFO:teuthology.orchestra.run.smithi092.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T20:56:51.572 INFO:teuthology.orchestra.run.smithi066.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T20:56:51.574 INFO:teuthology.orchestra.run.smithi066.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T20:56:51.575 INFO:teuthology.orchestra.run.smithi066.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T20:56:51.579 INFO:teuthology.orchestra.run.smithi092.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T20:56:51.579 INFO:teuthology.orchestra.run.smithi066.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T20:56:51.581 INFO:teuthology.orchestra.run.smithi027.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T20:56:51.581 INFO:teuthology.orchestra.run.smithi092.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T20:56:51.581 INFO:teuthology.orchestra.run.smithi092.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T20:56:51.582 INFO:teuthology.orchestra.run.smithi066.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T20:56:51.583 INFO:teuthology.orchestra.run.smithi066.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T20:56:51.583 INFO:teuthology.orchestra.run.smithi066.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T20:56:51.584 INFO:teuthology.orchestra.run.smithi027.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T20:56:51.584 INFO:teuthology.orchestra.run.smithi092.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T20:56:51.585 INFO:teuthology.orchestra.run.smithi092.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T20:56:51.586 INFO:teuthology.orchestra.run.smithi027.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T20:56:51.586 INFO:teuthology.orchestra.run.smithi027.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T20:56:51.586 INFO:teuthology.orchestra.run.smithi027.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T20:56:51.588 INFO:teuthology.orchestra.run.smithi092.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T20:56:51.588 INFO:teuthology.orchestra.run.smithi092.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T20:56:51.588 INFO:teuthology.orchestra.run.smithi092.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T20:56:51.589 INFO:teuthology.orchestra.run.smithi066.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T20:56:51.590 INFO:teuthology.orchestra.run.smithi092.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T20:56:51.590 INFO:teuthology.orchestra.run.smithi066.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T20:56:51.591 INFO:teuthology.orchestra.run.smithi092.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T20:56:51.593 INFO:teuthology.orchestra.run.smithi066.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T20:56:51.595 INFO:teuthology.orchestra.run.smithi092.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T20:56:51.596 INFO:teuthology.orchestra.run.smithi092.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T20:56:51.597 INFO:teuthology.orchestra.run.smithi092.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T20:56:51.597 INFO:teuthology.orchestra.run.smithi092.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T20:56:51.598 INFO:teuthology.orchestra.run.smithi027.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T20:56:51.598 INFO:teuthology.orchestra.run.smithi027.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T20:56:51.601 INFO:teuthology.orchestra.run.smithi092.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T20:56:51.602 INFO:teuthology.orchestra.run.smithi027.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T20:56:51.602 INFO:teuthology.orchestra.run.smithi027.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T20:56:51.602 INFO:teuthology.orchestra.run.smithi027.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T20:56:51.606 INFO:teuthology.orchestra.run.smithi066.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T20:56:51.607 INFO:teuthology.orchestra.run.smithi066.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T20:56:51.607 INFO:teuthology.orchestra.run.smithi066.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T20:56:51.609 INFO:teuthology.orchestra.run.smithi066.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T20:56:51.609 INFO:teuthology.orchestra.run.smithi066.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T20:56:51.614 INFO:teuthology.orchestra.run.smithi092.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T20:56:51.614 INFO:teuthology.orchestra.run.smithi092.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T20:56:51.614 INFO:teuthology.orchestra.run.smithi092.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T20:56:51.615 INFO:teuthology.orchestra.run.smithi027.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T20:56:51.615 INFO:teuthology.orchestra.run.smithi027.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T20:56:51.616 INFO:teuthology.orchestra.run.smithi066.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T20:56:51.616 INFO:teuthology.orchestra.run.smithi066.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T20:56:51.616 INFO:teuthology.orchestra.run.smithi092.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T20:56:51.616 INFO:teuthology.orchestra.run.smithi092.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T20:56:51.618 INFO:teuthology.orchestra.run.smithi027.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T20:56:51.618 INFO:teuthology.orchestra.run.smithi027.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T20:56:51.619 INFO:teuthology.orchestra.run.smithi027.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T20:56:51.622 INFO:teuthology.orchestra.run.smithi092.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T20:56:51.622 INFO:teuthology.orchestra.run.smithi092.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T20:56:51.623 INFO:teuthology.orchestra.run.smithi066.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T20:56:51.625 INFO:teuthology.orchestra.run.smithi066.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T20:56:51.625 INFO:teuthology.orchestra.run.smithi066.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T20:56:51.630 INFO:teuthology.orchestra.run.smithi092.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T20:56:51.632 INFO:teuthology.orchestra.run.smithi092.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T20:56:51.633 INFO:teuthology.orchestra.run.smithi027.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T20:56:51.633 INFO:teuthology.orchestra.run.smithi092.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T20:56:51.634 INFO:teuthology.orchestra.run.smithi027.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T20:56:51.634 INFO:teuthology.orchestra.run.smithi027.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T20:56:51.634 INFO:teuthology.orchestra.run.smithi027.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T20:56:51.635 INFO:teuthology.orchestra.run.smithi027.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T20:56:51.641 INFO:teuthology.orchestra.run.smithi092.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T20:56:51.650 INFO:teuthology.orchestra.run.smithi027.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T20:56:51.651 INFO:teuthology.orchestra.run.smithi027.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T20:56:51.656 INFO:teuthology.orchestra.run.smithi027.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T20:56:51.656 INFO:teuthology.orchestra.run.smithi027.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T20:56:51.657 INFO:teuthology.orchestra.run.smithi027.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T20:56:51.667 INFO:teuthology.orchestra.run.smithi027.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T20:56:51.669 INFO:teuthology.orchestra.run.smithi027.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T20:56:51.671 INFO:teuthology.orchestra.run.smithi027.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T20:56:51.671 INFO:teuthology.orchestra.run.smithi027.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T20:56:51.673 INFO:teuthology.orchestra.run.smithi027.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T20:56:51.685 INFO:teuthology.orchestra.run.smithi027.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T20:56:51.689 INFO:teuthology.orchestra.run.smithi027.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T20:56:51.689 INFO:teuthology.orchestra.run.smithi027.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T20:56:51.690 INFO:teuthology.orchestra.run.smithi027.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T20:56:51.693 INFO:teuthology.orchestra.run.smithi027.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T20:56:51.720 INFO:teuthology.orchestra.run.smithi066.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T20:56:51.747 INFO:teuthology.orchestra.run.smithi027.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T20:56:51.748 INFO:teuthology.orchestra.run.smithi027.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T20:56:51.749 INFO:teuthology.orchestra.run.smithi027.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T20:56:51.749 INFO:teuthology.orchestra.run.smithi027.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T20:56:51.750 INFO:teuthology.orchestra.run.smithi027.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T20:56:51.752 INFO:teuthology.orchestra.run.smithi066.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T20:56:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T20:56:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T20:56:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T20:56:51.785 INFO:teuthology.orchestra.run.smithi027.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T20:56:51.786 INFO:teuthology.orchestra.run.smithi027.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T20:56:51.786 INFO:teuthology.orchestra.run.smithi027.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T20:56:51.786 INFO:teuthology.orchestra.run.smithi027.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T20:56:51.787 INFO:teuthology.orchestra.run.smithi027.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T20:56:51.787 INFO:teuthology.orchestra.run.smithi027.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T20:56:51.801 INFO:teuthology.orchestra.run.smithi027.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T20:56:51.801 INFO:teuthology.orchestra.run.smithi027.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T20:56:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T20:56:51.818 INFO:teuthology.orchestra.run.smithi027.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T20:56:51.819 INFO:teuthology.orchestra.run.smithi027.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T20:56:51.820 INFO:teuthology.orchestra.run.smithi027.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T20:56:51.823 INFO:teuthology.orchestra.run.smithi027.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T20:56:51.825 INFO:teuthology.orchestra.run.smithi027.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T20:56:51.827 INFO:teuthology.orchestra.run.smithi027.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T20:56:51.827 INFO:teuthology.orchestra.run.smithi027.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T20:56:51.836 INFO:teuthology.orchestra.run.smithi027.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T20:56:51.836 INFO:teuthology.orchestra.run.smithi027.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T20:56:51.839 INFO:teuthology.orchestra.run.smithi027.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T20:56:51.841 INFO:teuthology.orchestra.run.smithi066.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T20:56:51.853 INFO:teuthology.orchestra.run.smithi027.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T20:56:51.853 INFO:teuthology.orchestra.run.smithi027.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T20:56:51.854 INFO:teuthology.orchestra.run.smithi027.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T20:56:51.855 INFO:teuthology.orchestra.run.smithi027.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T20:56:51.856 INFO:teuthology.orchestra.run.smithi027.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T20:56:51.862 INFO:teuthology.orchestra.run.smithi027.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T20:56:51.862 INFO:teuthology.orchestra.run.smithi027.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T20:56:51.870 INFO:teuthology.orchestra.run.smithi027.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T20:56:51.872 INFO:teuthology.orchestra.run.smithi027.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T20:56:51.873 INFO:teuthology.orchestra.run.smithi027.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T20:56:51.896 INFO:teuthology.orchestra.run.smithi092.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T20:56:51.898 INFO:teuthology.orchestra.run.smithi092.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T20:56:51.907 INFO:teuthology.orchestra.run.smithi092.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T20:56:52.766 INFO:teuthology.orchestra.run.smithi092.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T20:56:52.846 INFO:teuthology.orchestra.run.smithi092.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T20:56:52.848 INFO:teuthology.orchestra.run.smithi092.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T20:56:52.850 INFO:teuthology.orchestra.run.smithi092.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T20:56:52.888 INFO:teuthology.orchestra.run.smithi092.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T20:56:53.040 INFO:teuthology.orchestra.run.smithi092.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T20:56:53.123 INFO:teuthology.orchestra.run.smithi027.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T20:56:53.190 INFO:teuthology.orchestra.run.smithi066.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T20:56:53.411 INFO:teuthology.orchestra.run.smithi066.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T20:56:53.416 INFO:teuthology.orchestra.run.smithi066.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T20:56:53.420 INFO:teuthology.orchestra.run.smithi066.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T20:56:53.524 INFO:teuthology.orchestra.run.smithi066.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T20:56:53.671 INFO:teuthology.orchestra.run.smithi092.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T20:56:53.672 INFO:teuthology.orchestra.run.smithi092.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T20:56:53.686 INFO:teuthology.orchestra.run.smithi092.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T20:56:53.723 INFO:teuthology.orchestra.run.smithi092.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T20:56:53.733 INFO:teuthology.orchestra.run.smithi092.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T20:56:53.735 INFO:teuthology.orchestra.run.smithi092.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T20:56:53.822 INFO:teuthology.orchestra.run.smithi027.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T20:56:53.889 INFO:teuthology.orchestra.run.smithi066.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T20:56:53.920 INFO:teuthology.orchestra.run.smithi027.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T20:56:53.951 INFO:teuthology.orchestra.run.smithi027.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T20:56:53.952 INFO:teuthology.orchestra.run.smithi027.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T20:56:54.001 INFO:teuthology.orchestra.run.smithi027.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T20:56:54.002 INFO:teuthology.orchestra.run.smithi027.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T20:56:54.029 INFO:teuthology.orchestra.run.smithi027.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T20:56:54.910 INFO:teuthology.orchestra.run.smithi027.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T20:56:54.921 INFO:teuthology.orchestra.run.smithi027.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T20:56:55.028 INFO:teuthology.orchestra.run.smithi027.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T20:56:55.326 INFO:teuthology.orchestra.run.smithi066.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T20:56:55.327 INFO:teuthology.orchestra.run.smithi066.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T20:56:55.344 INFO:teuthology.orchestra.run.smithi066.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T20:56:55.475 INFO:teuthology.orchestra.run.smithi066.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T20:56:55.505 INFO:teuthology.orchestra.run.smithi066.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T20:56:55.536 INFO:teuthology.orchestra.run.smithi066.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T20:56:55.971 INFO:teuthology.orchestra.run.smithi092.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T20:56:56.164 INFO:teuthology.orchestra.run.smithi092.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T20:56:56.165 INFO:teuthology.orchestra.run.smithi092.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T20:56:57.279 INFO:teuthology.orchestra.run.smithi027.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T20:56:57.641 INFO:teuthology.orchestra.run.smithi027.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T20:56:57.646 INFO:teuthology.orchestra.run.smithi027.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T20:56:57.674 INFO:teuthology.orchestra.run.smithi027.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T20:56:57.782 INFO:teuthology.orchestra.run.smithi027.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T20:56:58.072 INFO:teuthology.orchestra.run.smithi092.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T20:56:58.075 INFO:teuthology.orchestra.run.smithi092.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T20:56:58.076 INFO:teuthology.orchestra.run.smithi092.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T20:56:58.163 INFO:teuthology.orchestra.run.smithi027.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T20:56:58.586 INFO:teuthology.orchestra.run.smithi092.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T20:56:59.153 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 244 MB in 8s (32.4 MB/s) 2024-09-15T20:56:59.182 INFO:teuthology.orchestra.run.smithi066.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T20:56:59.288 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T20:56:59.306 INFO:teuthology.orchestra.run.smithi027.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T20:56:59.307 INFO:teuthology.orchestra.run.smithi027.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T20:56:59.324 INFO:teuthology.orchestra.run.smithi027.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T20:56:59.327 INFO:teuthology.orchestra.run.smithi066.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T20:56:59.328 INFO:teuthology.orchestra.run.smithi066.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T20:56:59.345 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T20:56:59.350 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T20:56:59.379 INFO:teuthology.orchestra.run.smithi027.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T20:56:59.415 INFO:teuthology.orchestra.run.smithi027.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T20:56:59.419 INFO:teuthology.orchestra.run.smithi027.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T20:56:59.429 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:56:59.758 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T20:56:59.784 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T20:56:59.824 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:57:00.161 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T20:57:00.188 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T20:57:00.227 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:57:00.589 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T20:57:00.616 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:00.656 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:01.059 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T20:57:01.085 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:01.125 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:01.272 INFO:teuthology.orchestra.run.smithi066.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T20:57:01.274 INFO:teuthology.orchestra.run.smithi066.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T20:57:01.275 INFO:teuthology.orchestra.run.smithi066.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T20:57:01.470 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T20:57:01.497 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:01.537 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:01.788 INFO:teuthology.orchestra.run.smithi066.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T20:57:01.849 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T20:57:01.864 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T20:57:01.898 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:57:02.162 INFO:teuthology.orchestra.run.smithi027.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T20:57:02.192 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librados2. 2024-09-15T20:57:02.208 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:02.242 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:02.294 INFO:teuthology.orchestra.run.smithi027.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T20:57:02.294 INFO:teuthology.orchestra.run.smithi027.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T20:57:02.384 INFO:teuthology.orchestra.run.smithi066.stdout:Fetched 244 MB in 11s (22.7 MB/s) 2024-09-15T20:57:02.515 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T20:57:02.597 INFO:teuthology.orchestra.run.smithi066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T20:57:02.602 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T20:57:02.673 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:57:02.721 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libnbd0. 2024-09-15T20:57:02.747 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T20:57:02.787 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T20:57:02.993 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T20:57:03.019 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T20:57:03.059 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:57:03.115 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librbd1. 2024-09-15T20:57:03.141 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:03.182 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:03.412 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T20:57:03.439 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T20:57:03.479 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:57:03.643 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T20:57:03.669 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T20:57:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T20:57:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T20:57:03.709 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:03.916 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T20:57:03.943 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:03.991 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:04.004 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rados. 2024-09-15T20:57:04.030 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:04.066 INFO:teuthology.orchestra.run.smithi027.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T20:57:04.071 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:04.382 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T20:57:04.385 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T20:57:04.408 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:04.412 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:04.448 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:04.452 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:04.658 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T20:57:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:Fetched 244 MB in 13s (19.0 MB/s) 2024-09-15T20:57:04.674 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:04.708 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:04.813 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T20:57:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T20:57:04.840 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:04.880 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:04.920 INFO:teuthology.orchestra.run.smithi027.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T20:57:04.926 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T20:57:05.012 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T20:57:05.029 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:57:05.038 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:05.078 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:05.199 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T20:57:05.227 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T20:57:05.258 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:57:05.398 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T20:57:05.424 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T20:57:05.425 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T20:57:05.451 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T20:57:05.464 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:57:05.482 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:57:05.577 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librados2. 2024-09-15T20:57:05.604 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:05.644 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:05.817 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T20:57:05.835 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T20:57:05.843 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T20:57:05.862 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T20:57:05.884 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T20:57:05.902 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:57:06.120 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T20:57:06.135 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:06.138 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libnbd0. 2024-09-15T20:57:06.165 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T20:57:06.170 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:06.205 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T20:57:06.356 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T20:57:06.382 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:06.430 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:06.516 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librbd1. 2024-09-15T20:57:06.543 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:06.582 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:06.748 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T20:57:06.774 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T20:57:06.808 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T20:57:06.814 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:57:06.824 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:06.858 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:06.977 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T20:57:07.004 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:07.035 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:07.101 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T20:57:07.127 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T20:57:07.167 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:57:07.387 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T20:57:07.413 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T20:57:07.446 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rados. 2024-09-15T20:57:07.453 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:57:07.462 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:07.479 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T20:57:07.504 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:07.505 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T20:57:07.545 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:57:07.823 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T20:57:07.832 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua5.1. 2024-09-15T20:57:07.832 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T20:57:07.847 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T20:57:07.849 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T20:57:07.859 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:07.889 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:57:07.890 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:57:07.898 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:08.151 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T20:57:08.159 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package librados2. 2024-09-15T20:57:08.178 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:08.183 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:08.201 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-any. 2024-09-15T20:57:08.216 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T20:57:08.217 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:08.217 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:08.259 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T20:57:08.512 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package zip. 2024-09-15T20:57:08.527 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T20:57:08.537 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T20:57:08.564 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:08.570 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T20:57:08.605 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:08.720 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libnbd0. 2024-09-15T20:57:08.747 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T20:57:08.786 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T20:57:08.832 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package unzip. 2024-09-15T20:57:08.858 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T20:57:08.890 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:57:08.923 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T20:57:08.950 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T20:57:08.990 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:57:09.106 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package librbd1. 2024-09-15T20:57:09.132 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:09.172 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:09.293 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T20:57:09.295 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package luarocks. 2024-09-15T20:57:09.332 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T20:57:09.332 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T20:57:09.359 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T20:57:09.361 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:57:09.609 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T20:57:09.619 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T20:57:09.625 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:09.636 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:09.659 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:09.678 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:09.744 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2. 2024-09-15T20:57:09.770 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:09.818 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:09.920 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-rados. 2024-09-15T20:57:09.935 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:09.978 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:10.006 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T20:57:10.033 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T20:57:10.064 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:57:10.306 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T20:57:10.332 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:10.366 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T20:57:10.372 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:10.394 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T20:57:10.413 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T20:57:10.428 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:10.433 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:57:10.463 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:10.624 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T20:57:10.640 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:10.683 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:10.736 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T20:57:10.763 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T20:57:10.774 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T20:57:10.800 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T20:57:10.802 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:57:10.840 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:57:10.969 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T20:57:10.996 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:11.035 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:11.139 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua5.1. 2024-09-15T20:57:11.152 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T20:57:11.167 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T20:57:11.178 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:11.205 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:57:11.218 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:11.322 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T20:57:11.348 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T20:57:11.388 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:57:11.500 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-any. 2024-09-15T20:57:11.527 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T20:57:11.567 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T20:57:11.605 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common. 2024-09-15T20:57:11.631 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:11.671 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:11.708 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T20:57:11.735 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T20:57:11.774 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T20:57:11.819 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package zip. 2024-09-15T20:57:11.845 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T20:57:11.886 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T20:57:12.044 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T20:57:12.059 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:12.102 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:12.205 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package unzip. 2024-09-15T20:57:12.233 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T20:57:12.272 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:57:12.605 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T20:57:12.632 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T20:57:12.663 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:57:12.776 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package luarocks. 2024-09-15T20:57:12.792 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T20:57:12.834 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:57:12.917 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T20:57:12.943 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T20:57:12.982 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:57:13.183 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package librgw2. 2024-09-15T20:57:13.210 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:13.227 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-base. 2024-09-15T20:57:13.241 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:13.253 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:13.285 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T20:57:13.303 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:13.312 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T20:57:13.352 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:57:13.689 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package lua5.1. 2024-09-15T20:57:13.715 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T20:57:13.755 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:57:13.852 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T20:57:13.879 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:13.918 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:13.965 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T20:57:13.991 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T20:57:14.025 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package lua-any. 2024-09-15T20:57:14.039 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:57:14.052 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T20:57:14.083 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T20:57:14.288 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T20:57:14.319 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T20:57:14.329 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package zip. 2024-09-15T20:57:14.354 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T20:57:14.355 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:57:14.360 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T20:57:14.385 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T20:57:14.394 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T20:57:14.426 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:57:14.649 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T20:57:14.677 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:14.714 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package unzip. 2024-09-15T20:57:14.716 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:14.741 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T20:57:14.746 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T20:57:14.772 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T20:57:14.780 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:57:14.812 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:57:15.061 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-common. 2024-09-15T20:57:15.088 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:15.127 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:15.159 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package luarocks. 2024-09-15T20:57:15.165 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T20:57:15.186 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T20:57:15.191 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T20:57:15.217 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:57:15.240 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:57:15.569 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package librgw2. 2024-09-15T20:57:15.596 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:15.609 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T20:57:15.635 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:15.635 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T20:57:15.684 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:57:15.996 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T20:57:16.022 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T20:57:16.062 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T20:57:16.254 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T20:57:16.281 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:16.401 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-portend. 2024-09-15T20:57:16.424 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T20:57:16.473 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T20:57:16.632 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T20:57:16.659 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T20:57:16.690 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:57:16.707 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-base. 2024-09-15T20:57:16.734 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:16.784 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:16.785 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T20:57:16.811 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T20:57:16.851 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T20:57:16.993 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T20:57:17.020 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:17.060 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:17.163 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T20:57:17.189 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T20:57:17.229 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:57:17.336 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T20:57:17.363 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T20:57:17.402 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:57:17.489 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-common. 2024-09-15T20:57:17.515 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:17.555 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:17.599 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T20:57:17.628 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T20:57:17.666 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T20:57:17.881 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T20:57:17.907 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T20:57:17.947 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:57:18.225 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T20:57:18.244 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T20:57:18.252 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T20:57:18.271 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T20:57:18.283 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:57:18.319 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T20:57:18.607 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T20:57:18.614 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T20:57:18.634 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T20:57:18.640 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T20:57:18.677 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:57:18.680 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:57:18.980 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T20:57:19.000 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-mako. 2024-09-15T20:57:19.007 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T20:57:19.026 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T20:57:19.047 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:57:19.074 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:57:19.102 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-base. 2024-09-15T20:57:19.129 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:19.170 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:19.325 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T20:57:19.351 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T20:57:19.383 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T20:57:19.453 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T20:57:19.479 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T20:57:19.528 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:57:19.652 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-portend. 2024-09-15T20:57:19.679 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T20:57:19.719 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T20:57:19.773 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T20:57:19.800 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T20:57:19.839 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:57:19.898 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T20:57:19.924 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T20:57:19.972 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:57:20.030 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T20:57:20.057 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T20:57:20.097 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T20:57:20.126 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T20:57:20.153 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T20:57:20.217 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:57:20.283 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webob. 2024-09-15T20:57:20.310 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T20:57:20.350 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:57:20.425 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T20:57:20.452 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T20:57:20.496 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:57:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T20:57:20.573 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T20:57:20.612 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:57:20.720 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T20:57:20.746 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T20:57:20.790 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:57:20.811 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T20:57:20.838 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T20:57:20.877 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T20:57:20.915 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T20:57:20.942 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T20:57:20.981 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:57:21.098 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T20:57:21.124 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T20:57:21.148 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T20:57:21.164 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:57:21.174 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T20:57:21.206 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T20:57:21.259 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T20:57:21.286 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T20:57:21.317 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:57:21.467 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-paste. 2024-09-15T20:57:21.493 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T20:57:21.508 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T20:57:21.538 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T20:57:21.538 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:57:21.575 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:57:21.604 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T20:57:21.631 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T20:57:21.670 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T20:57:21.862 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T20:57:21.877 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-mako. 2024-09-15T20:57:21.888 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T20:57:21.904 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T20:57:21.928 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:57:21.944 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:57:21.973 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-portend. 2024-09-15T20:57:22.000 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T20:57:22.040 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T20:57:22.256 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T20:57:22.281 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T20:57:22.282 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T20:57:22.308 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T20:57:22.323 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:57:22.343 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T20:57:22.347 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:57:22.370 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T20:57:22.409 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T20:57:22.626 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T20:57:22.652 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T20:57:22.692 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T20:57:22.696 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T20:57:22.723 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T20:57:22.762 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:57:22.987 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T20:57:23.014 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T20:57:23.054 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:57:23.358 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T20:57:23.385 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T20:57:23.424 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T20:57:23.509 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T20:57:23.525 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T20:57:23.567 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:57:23.686 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T20:57:23.702 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T20:57:23.736 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T20:57:23.870 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webob. 2024-09-15T20:57:23.897 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T20:57:23.936 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:57:24.022 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T20:57:24.049 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T20:57:24.088 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:57:24.226 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T20:57:24.253 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T20:57:24.289 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T20:57:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T20:57:24.359 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:57:24.501 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T20:57:24.717 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T20:57:24.745 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T20:57:24.792 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:57:24.838 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-idna. 2024-09-15T20:57:24.864 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T20:57:24.904 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:57:25.142 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-mako. 2024-09-15T20:57:25.161 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-paste. 2024-09-15T20:57:25.169 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T20:57:25.173 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T20:57:25.189 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T20:57:25.200 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T20:57:25.208 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:57:25.231 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:57:25.236 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:57:25.545 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T20:57:25.551 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests. 2024-09-15T20:57:25.568 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T20:57:25.572 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T20:57:25.578 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T20:57:25.594 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T20:57:25.618 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:57:25.641 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:57:25.645 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:57:25.915 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T20:57:25.929 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T20:57:25.941 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T20:57:25.956 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T20:57:25.969 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T20:57:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:57:25.996 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:57:25.996 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T20:57:26.036 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:57:26.267 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-webob. 2024-09-15T20:57:26.294 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T20:57:26.307 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T20:57:26.323 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:26.333 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:57:26.338 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T20:57:26.366 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T20:57:26.366 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:26.405 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T20:57:26.712 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T20:57:26.720 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T20:57:26.728 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:26.747 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T20:57:26.758 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T20:57:26.762 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:26.785 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T20:57:26.790 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:57:26.833 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:57:27.015 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T20:57:27.042 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T20:57:27.082 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:57:27.098 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T20:57:27.125 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T20:57:27.165 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:57:27.187 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T20:57:27.215 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T20:57:27.254 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T20:57:27.318 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T20:57:27.345 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T20:57:27.384 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T20:57:27.467 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-paste. 2024-09-15T20:57:27.495 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T20:57:27.534 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:57:27.573 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-idna. 2024-09-15T20:57:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T20:57:27.640 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:57:27.730 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T20:57:27.756 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:27.796 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:27.879 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T20:57:27.895 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T20:57:27.937 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:57:27.968 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T20:57:27.995 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T20:57:28.034 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:57:28.208 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T20:57:28.215 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T20:57:28.234 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:28.242 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T20:57:28.274 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:28.282 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:57:28.488 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests. 2024-09-15T20:57:28.515 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T20:57:28.551 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T20:57:28.554 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:57:28.578 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T20:57:28.618 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T20:57:28.865 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T20:57:28.892 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T20:57:28.929 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T20:57:28.931 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:57:28.956 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T20:57:28.986 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T20:57:28.996 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:57:29.013 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:29.044 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:29.210 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T20:57:29.226 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:29.268 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:29.351 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T20:57:29.378 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T20:57:29.417 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T20:57:29.614 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T20:57:29.630 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:29.664 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:29.762 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-idna. 2024-09-15T20:57:29.789 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T20:57:29.828 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:57:29.967 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T20:57:29.997 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T20:57:30.036 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:57:30.097 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph. 2024-09-15T20:57:30.113 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:30.132 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T20:57:30.147 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:30.159 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T20:57:30.190 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:57:30.337 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T20:57:30.363 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T20:57:30.403 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T20:57:30.433 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T20:57:30.459 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:30.476 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-requests. 2024-09-15T20:57:30.491 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:30.503 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T20:57:30.534 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:57:30.747 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T20:57:30.775 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:30.814 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:30.861 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T20:57:30.888 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:30.913 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T20:57:30.927 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:30.940 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T20:57:30.988 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:57:31.275 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T20:57:31.299 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T20:57:31.303 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:31.315 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:31.340 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephadm. 2024-09-15T20:57:31.342 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:31.357 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:31.365 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:31.405 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:31.691 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T20:57:31.707 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T20:57:31.741 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T20:57:31.745 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T20:57:31.773 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:31.812 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:31.962 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T20:57:31.989 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:32.028 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:32.078 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T20:57:32.104 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:32.132 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T20:57:32.144 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:32.159 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T20:57:32.198 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:57:32.501 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T20:57:32.528 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T20:57:32.567 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T20:57:32.871 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T20:57:32.898 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:32.929 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:33.156 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph. 2024-09-15T20:57:33.165 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-babel. 2024-09-15T20:57:33.181 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:33.184 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:33.215 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:33.231 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:33.341 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T20:57:33.368 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:33.410 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:33.518 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T20:57:33.545 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T20:57:33.577 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:57:33.651 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T20:57:33.678 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:33.725 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:34.038 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T20:57:34.065 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:34.103 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T20:57:34.104 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:34.131 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:34.135 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T20:57:34.163 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:34.170 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:34.202 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:34.450 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T20:57:34.477 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T20:57:34.508 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T20:57:34.565 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package cephadm. 2024-09-15T20:57:34.591 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:34.631 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:34.811 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-routes. 2024-09-15T20:57:34.838 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T20:57:34.877 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:57:35.009 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T20:57:35.036 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T20:57:35.083 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T20:57:35.222 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T20:57:35.249 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:35.289 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:35.289 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph. 2024-09-15T20:57:35.305 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:35.339 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:35.436 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T20:57:35.453 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:35.503 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:35.658 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T20:57:35.685 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:35.724 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:36.094 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T20:57:36.121 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:36.169 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:36.539 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package cephadm. 2024-09-15T20:57:36.541 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-babel. 2024-09-15T20:57:36.555 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:36.557 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:36.588 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:36.599 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:36.875 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T20:57:36.902 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T20:57:36.911 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T20:57:36.938 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T20:57:36.941 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T20:57:36.977 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:57:37.269 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T20:57:37.280 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T20:57:37.296 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:37.297 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:37.327 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:37.330 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:37.642 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T20:57:37.669 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T20:57:37.700 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T20:57:38.003 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-routes. 2024-09-15T20:57:38.030 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T20:57:38.069 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:57:38.195 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T20:57:38.213 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T20:57:38.254 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:57:38.358 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-babel. 2024-09-15T20:57:38.385 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T20:57:38.424 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:57:38.431 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T20:57:38.458 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:38.505 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:38.711 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T20:57:38.738 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T20:57:38.766 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T20:57:38.768 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:57:38.793 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T20:57:38.832 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:57:39.097 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T20:57:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:39.155 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:39.194 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T20:57:39.221 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T20:57:39.260 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:57:39.483 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T20:57:39.499 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T20:57:39.533 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T20:57:39.538 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T20:57:39.566 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T20:57:39.690 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:57:39.803 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-routes. 2024-09-15T20:57:39.830 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T20:57:39.869 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:57:40.241 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T20:57:40.479 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:40.480 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T20:57:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:40.480 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:41.369 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T20:57:41.396 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T20:57:41.435 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T20:57:41.552 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T20:57:41.570 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T20:57:41.610 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:57:41.747 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T20:57:41.774 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T20:57:41.813 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:57:42.088 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T20:57:42.116 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T20:57:42.154 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:57:42.166 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T20:57:42.194 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T20:57:42.233 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:57:42.525 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T20:57:42.553 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T20:57:42.561 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T20:57:42.588 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T20:57:42.590 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:57:42.627 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T20:57:42.877 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T20:57:42.905 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T20:57:42.914 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T20:57:42.942 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T20:57:42.972 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T20:57:42.994 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T20:57:43.012 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T20:57:43.020 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:57:43.052 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:57:43.251 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T20:57:43.264 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T20:57:43.300 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:57:43.779 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T20:57:43.807 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T20:57:43.853 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T20:57:43.915 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T20:57:43.921 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T20:57:43.943 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T20:57:43.950 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:43.981 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:57:43.987 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:44.324 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T20:57:44.352 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T20:57:44.359 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T20:57:44.387 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T20:57:44.390 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:57:44.426 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:57:44.737 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T20:57:44.766 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T20:57:44.906 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:57:44.966 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T20:57:44.983 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T20:57:45.015 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T20:57:45.087 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T20:57:45.115 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:45.145 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:45.293 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T20:57:45.322 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T20:57:45.360 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:57:45.448 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T20:57:45.476 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T20:57:45.514 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:57:45.615 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T20:57:45.645 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:45.682 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:45.697 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T20:57:45.725 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T20:57:45.763 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:57:45.842 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T20:57:45.870 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:45.909 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:57:46.074 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T20:57:46.102 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T20:57:46.140 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T20:57:46.178 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package jq. 2024-09-15T20:57:46.206 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:46.244 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T20:57:46.453 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T20:57:46.472 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package socat. 2024-09-15T20:57:46.481 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T20:57:46.488 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T20:57:46.519 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T20:57:46.522 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:57:46.643 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T20:57:46.672 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T20:57:46.712 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T20:57:46.839 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T20:57:46.859 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T20:57:46.867 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T20:57:46.886 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T20:57:46.905 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:57:46.925 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:57:47.005 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T20:57:47.033 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T20:57:47.062 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:57:47.175 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T20:57:47.200 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T20:57:47.242 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T20:57:47.313 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test. 2024-09-15T20:57:47.333 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T20:57:47.341 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:47.361 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T20:57:47.379 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:47.391 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:57:47.545 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T20:57:47.573 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T20:57:47.611 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:57:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T20:57:47.669 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T20:57:47.710 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T20:57:48.031 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T20:57:48.060 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T20:57:48.097 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T20:57:48.225 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T20:57:48.242 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:48.274 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:48.443 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T20:57:48.471 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T20:57:48.509 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:57:48.603 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T20:57:48.631 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T20:57:48.669 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:57:48.804 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T20:57:48.820 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T20:57:48.863 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T20:57:49.141 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T20:57:49.169 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T20:57:49.198 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:57:49.230 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T20:57:49.259 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:49.297 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:57:49.566 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package jq. 2024-09-15T20:57:49.567 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T20:57:49.583 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:49.595 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:49.624 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T20:57:49.633 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:49.871 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T20:57:49.877 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package socat. 2024-09-15T20:57:49.889 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:49.894 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T20:57:49.929 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:49.929 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T20:57:49.935 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:57:49.957 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:49.995 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:50.230 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T20:57:50.247 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T20:57:50.249 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T20:57:50.265 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T20:57:50.277 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T20:57:50.279 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:57:50.293 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T20:57:50.315 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:57:50.331 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:57:50.584 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-test. 2024-09-15T20:57:50.613 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:50.647 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:50.653 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T20:57:50.669 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T20:57:50.681 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:50.698 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T20:57:50.718 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:57:50.769 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:57:51.029 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package jq. 2024-09-15T20:57:51.058 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T20:57:51.104 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T20:57:51.140 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T20:57:51.167 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T20:57:51.206 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:57:51.390 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package socat. 2024-09-15T20:57:51.420 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T20:57:51.465 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:57:51.628 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package pkg-config. 2024-09-15T20:57:51.656 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T20:57:51.694 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:57:51.793 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T20:57:51.810 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T20:57:51.876 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:57:51.930 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T20:57:51.947 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T20:57:51.980 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:57:52.273 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-test. 2024-09-15T20:57:52.290 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:52.339 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:52.392 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T20:57:52.420 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T20:57:52.450 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T20:57:52.736 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T20:57:52.764 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T20:57:52.802 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T20:57:52.938 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T20:57:52.955 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:53.123 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T20:57:53.151 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T20:57:53.189 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:57:53.310 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T20:57:53.325 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:53.366 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:53.500 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T20:57:53.528 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T20:57:53.566 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T20:57:53.677 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T20:57:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T20:57:53.727 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:57:53.911 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-py. 2024-09-15T20:57:53.939 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T20:57:53.977 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T20:57:54.040 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T20:57:54.069 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T20:57:54.106 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:57:54.306 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T20:57:54.335 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T20:57:54.372 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:57:54.477 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T20:57:54.506 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T20:57:54.543 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:57:54.610 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T20:57:54.639 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T20:57:54.668 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:54.852 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T20:57:54.880 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T20:57:54.918 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:57:54.933 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T20:57:54.949 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:54.957 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package pkg-config. 2024-09-15T20:57:54.982 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:54.985 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T20:57:55.023 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:57:55.179 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-toml. 2024-09-15T20:57:55.195 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T20:57:55.229 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T20:57:55.293 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T20:57:55.293 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T20:57:55.321 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T20:57:55.322 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T20:57:55.359 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:57:55.360 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:57:55.507 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T20:57:55.535 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T20:57:55.573 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:57:55.731 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T20:57:55.737 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T20:57:55.754 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T20:57:55.760 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T20:57:55.796 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T20:57:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:57:55.910 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T20:57:55.938 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T20:57:55.968 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:57:56.098 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T20:57:56.127 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T20:57:56.128 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T20:57:56.154 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T20:57:56.173 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T20:57:56.185 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:57:56.237 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T20:57:56.254 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T20:57:56.287 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:57:56.502 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T20:57:56.530 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T20:57:56.568 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:57:56.582 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package pkg-config. 2024-09-15T20:57:56.599 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T20:57:56.640 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:57:56.879 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T20:57:56.893 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T20:57:56.908 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T20:57:56.922 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T20:57:56.924 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw. 2024-09-15T20:57:56.945 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T20:57:56.953 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:56.959 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:57:57.008 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:57.290 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-py. 2024-09-15T20:57:57.318 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T20:57:57.365 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T20:57:57.371 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T20:57:57.400 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T20:57:57.437 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T20:57:57.693 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T20:57:57.722 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T20:57:57.741 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T20:57:57.769 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T20:57:57.785 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:57:57.807 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T20:57:57.877 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T20:57:57.905 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:57:57.943 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:58.127 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T20:57:58.156 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T20:57:58.193 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:57:58.222 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T20:57:58.251 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T20:57:58.265 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T20:57:58.280 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:57:58.471 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T20:57:58.500 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T20:57:58.507 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:57:58.529 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T20:57:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-toml. 2024-09-15T20:57:58.625 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:57:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T20:57:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T20:57:58.866 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-py. 2024-09-15T20:57:58.884 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T20:57:58.895 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T20:57:58.932 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T20:57:58.969 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T20:57:58.998 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T20:57:59.035 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:57:59.224 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T20:57:59.561 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T20:57:59.590 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T20:57:59.614 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T20:57:59.619 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:57:59.629 INFO:teuthology.orchestra.run.smithi092.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T20:57:59.642 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T20:57:59.671 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:57:59.852 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:57:59.941 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T20:57:59.970 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T20:58:00.007 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:58:00.041 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T20:58:00.069 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T20:58:00.106 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:58:00.418 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-toml. 2024-09-15T20:58:00.447 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T20:58:00.476 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user cephadm....done 2024-09-15T20:58:00.484 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T20:58:00.632 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:58:00.728 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package radosgw. 2024-09-15T20:58:00.757 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:58:00.779 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T20:58:00.786 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:00.808 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T20:58:00.848 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:58:00.908 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:58:01.167 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:58:01.199 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T20:58:01.228 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T20:58:01.266 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:58:01.294 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:58:01.535 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T20:58:01.552 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T20:58:01.562 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T20:58:01.593 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:58:01.739 INFO:teuthology.orchestra.run.smithi066.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T20:58:01.756 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:58:01.797 INFO:teuthology.orchestra.run.smithi066.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:01.813 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:58:01.939 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T20:58:02.110 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T20:58:02.240 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:58:02.339 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package radosgw. 2024-09-15T20:58:02.368 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:58:02.405 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:02.418 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:58:02.536 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:58:02.609 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T20:58:02.777 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T20:58:02.860 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T20:58:03.102 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T20:58:03.145 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:58:03.300 INFO:teuthology.orchestra.run.smithi027.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T20:58:03.329 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T20:58:03.358 INFO:teuthology.orchestra.run.smithi027.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:03.396 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:58:03.521 INFO:teuthology.orchestra.run.smithi066.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T20:58:03.596 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T20:58:03.647 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:58:03.898 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:58:03.955 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T20:58:04.071 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user cephadm....done 2024-09-15T20:58:04.089 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T20:58:04.158 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:04.236 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:58:04.386 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T20:58:04.417 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:58:04.537 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:58:04.543 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T20:58:04.677 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T20:58:04.788 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:58:04.803 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:58:04.905 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:58:04.914 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:58:05.139 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:58:05.164 INFO:teuthology.orchestra.run.smithi027.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T20:58:05.174 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T20:58:05.241 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:05.441 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:58:05.496 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T20:58:05.544 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T20:58:05.567 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T20:58:05.609 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:58:05.735 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T20:58:05.871 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:58:05.899 INFO:teuthology.orchestra.run.smithi027.stdout:Adding system user cephadm....done 2024-09-15T20:58:06.064 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T20:58:06.105 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:06.229 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T20:58:06.231 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T20:58:06.366 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T20:58:06.497 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:58:06.507 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T20:58:06.625 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T20:58:06.756 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:58:06.776 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T20:58:06.800 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:58:07.044 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T20:58:07.052 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:58:07.065 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T20:58:07.225 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:58:07.312 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T20:58:07.370 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:58:07.438 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T20:58:07.637 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:58:07.765 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T20:58:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:58:07.888 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:08.111 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:58:08.156 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:58:08.175 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T20:58:08.221 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:58:08.282 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T20:58:08.322 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:58:08.449 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:58:08.451 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T20:58:08.576 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:58:08.606 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T20:58:08.701 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:58:08.728 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T20:58:08.961 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:58:08.988 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T20:58:08.990 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T20:58:09.250 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:58:09.255 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T20:58:09.334 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T20:58:09.367 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T20:58:09.447 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:58:09.532 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T20:58:09.573 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T20:58:09.593 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:58:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:58:09.916 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:09.949 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:10.075 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T20:58:10.200 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T20:58:10.246 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:58:10.326 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T20:58:10.351 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:58:10.452 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T20:58:10.480 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:58:10.579 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T20:58:10.606 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:10.619 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:58:10.688 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T20:58:10.933 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:58:10.937 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T20:58:10.947 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T20:58:11.063 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:58:11.250 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:58:11.842 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:58:11.843 INFO:teuthology.orchestra.run.smithi027.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T20:58:11.843 INFO:teuthology.orchestra.run.smithi027.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T20:58:11.843 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T20:58:11.843 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T20:58:11.974 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:58:11.994 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:12.100 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:58:12.112 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T20:58:12.243 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:58:12.371 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T20:58:12.473 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:58:12.552 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T20:58:12.648 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T20:58:12.716 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:12.825 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T20:58:12.962 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T20:58:12.974 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T20:58:13.076 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:58:13.125 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T20:58:13.202 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:58:13.204 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:58:13.304 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:58:13.580 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:58:13.589 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:58:13.831 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T20:58:13.844 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T20:58:13.890 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:58:14.082 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:14.112 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T20:58:14.208 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:58:14.233 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:58:14.237 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T20:58:14.364 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T20:58:14.501 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:58:14.502 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:58:14.627 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:14.690 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T20:58:14.761 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T20:58:15.003 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T20:58:15.035 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:58:15.100 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T20:58:15.254 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:58:15.376 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T20:58:15.629 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:58:15.769 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T20:58:15.797 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T20:58:16.065 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:58:16.070 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T20:58:16.191 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T20:58:16.405 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T20:58:16.458 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:58:16.576 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:58:16.664 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T20:58:16.799 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:16.818 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:58:16.861 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:58:17.070 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:17.125 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T20:58:17.137 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:58:17.204 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T20:58:17.388 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:58:17.463 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:58:17.468 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T20:58:17.514 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:58:17.589 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:58:17.640 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:17.715 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:58:17.728 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T20:58:17.992 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:58:18.025 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:58:18.092 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:58:18.284 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T20:58:18.494 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:58:18.535 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:18.661 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:58:18.707 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T20:58:18.737 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T20:58:18.863 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T20:58:18.988 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:58:18.992 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:19.114 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:19.143 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T20:58:19.222 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T20:58:19.240 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:58:19.411 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T20:58:19.473 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T20:58:19.537 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T20:58:19.557 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:58:19.663 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T20:58:19.741 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:58:19.842 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:58:19.990 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T20:58:20.000 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T20:58:20.118 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T20:58:20.244 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T20:58:20.268 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:58:20.283 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T20:58:20.419 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T20:58:20.503 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T20:58:20.576 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:20.703 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:58:20.726 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T20:58:20.829 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:58:20.848 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:58:21.045 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T20:58:21.091 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:58:21.105 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:58:21.287 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T20:58:21.417 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:58:21.479 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T20:58:21.555 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T20:58:21.675 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:58:21.754 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:58:21.800 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:58:21.815 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T20:58:21.926 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:22.015 INFO:teuthology.orchestra.run.smithi066.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T20:58:22.083 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T20:58:22.090 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:58:22.191 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:22.261 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:58:22.292 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:22.359 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T20:58:22.379 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:58:22.418 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:58:22.510 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T20:58:22.756 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:58:22.772 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:22.811 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T20:58:22.897 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:22.929 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T20:58:22.998 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T20:58:23.023 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T20:58:23.124 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T20:58:23.221 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T20:58:23.283 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:23.392 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:23.409 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:23.522 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T20:58:23.542 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:58:23.552 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:23.751 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:23.751 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:23.799 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T20:58:23.910 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:58:23.950 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T20:58:24.123 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T20:58:24.219 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:58:24.231 INFO:teuthology.orchestra.run.smithi027.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T20:58:24.253 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:24.371 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:24.486 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T20:58:24.497 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:24.538 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T20:58:24.595 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T20:58:24.623 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:58:24.663 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T20:58:24.923 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T20:58:24.972 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T20:58:25.239 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T20:58:25.293 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:58:25.374 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T20:58:25.577 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:58:25.633 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T20:58:25.760 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T20:58:25.784 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:25.910 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:25.979 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T20:58:26.037 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:26.094 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T20:58:26.163 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:26.264 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:58:26.289 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T20:58:26.608 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:58:26.730 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T20:58:26.759 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:26.781 INFO:teuthology.orchestra.run.smithi092.stdout:Adding group ceph....done 2024-09-15T20:58:26.856 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T20:58:26.910 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:27.061 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:58:27.149 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T20:58:27.150 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user ceph....done 2024-09-15T20:58:27.318 INFO:teuthology.orchestra.run.smithi092.stdout:Setting system user ceph properties....done 2024-09-15T20:58:27.333 INFO:teuthology.orchestra.run.smithi092.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T20:58:27.430 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:27.468 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T20:58:27.527 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T20:58:27.556 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:27.682 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T20:58:27.764 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T20:58:27.795 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T20:58:27.975 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.101 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.163 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T20:58:28.239 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.252 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.374 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.473 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T20:58:28.491 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:28.491 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:28.818 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T20:58:28.840 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:28.840 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:28.968 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:28.979 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.130 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.221 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T20:58:29.300 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.565 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.582 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.619 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T20:58:29.741 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:29.783 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:58:29.892 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T20:58:30.060 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:30.281 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:30.282 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:30.362 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:30.487 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:30.613 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:30.779 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:30.833 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:30.833 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T20:58:31.004 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:31.005 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:31.053 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.179 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.305 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.331 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.431 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.457 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.513 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.557 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.583 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:31.709 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T20:58:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:32.050 INFO:teuthology.orchestra.run.smithi066.stdout:Adding group ceph....done 2024-09-15T20:58:32.240 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:32.366 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:32.393 INFO:teuthology.orchestra.run.smithi066.stdout:Adding system user ceph....done 2024-09-15T20:58:32.517 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:32.561 INFO:teuthology.orchestra.run.smithi066.stdout:Setting system user ceph properties....done 2024-09-15T20:58:32.582 INFO:teuthology.orchestra.run.smithi066.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T20:58:32.709 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:32.709 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:32.732 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T20:58:32.962 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.063 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.077 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T20:58:33.168 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.189 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.294 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.324 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.420 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.450 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.558 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.572 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.884 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:33.933 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T20:58:33.951 INFO:teuthology.orchestra.run.smithi027.stdout:Adding group ceph....done 2024-09-15T20:58:34.041 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T20:58:34.186 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T20:58:34.328 INFO:teuthology.orchestra.run.smithi027.stdout:Adding system user ceph....done 2024-09-15T20:58:34.369 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:34.369 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:34.496 INFO:teuthology.orchestra.run.smithi027.stdout:Setting system user ceph properties....done 2024-09-15T20:58:34.510 INFO:teuthology.orchestra.run.smithi027.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T20:58:34.662 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T20:58:34.852 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:34.992 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T20:58:35.232 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T20:58:35.261 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T20:58:35.452 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:35.561 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:35.917 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T20:58:35.923 DEBUG:teuthology.parallel:result is None 2024-09-15T20:58:36.031 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:36.078 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:36.078 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T20:58:36.275 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:36.275 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:36.547 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:36.733 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:36.911 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T20:58:36.977 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:36.977 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:37.391 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:37.468 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:37.616 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:37.616 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T20:58:37.781 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:37.781 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:38.068 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:38.269 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:38.295 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:38.296 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T20:58:38.395 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:38.538 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:38.746 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:38.746 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:38.777 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.053 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:39.053 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T20:58:39.222 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.382 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.520 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.524 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.646 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.701 INFO:teuthology.orchestra.run.smithi066.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:39.798 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:40.026 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T20:58:40.103 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T20:58:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T20:58:40.365 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T20:58:40.449 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:40.635 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:40.761 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:40.912 INFO:teuthology.orchestra.run.smithi027.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T20:58:41.332 INFO:teuthology.orchestra.run.smithi027.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T20:58:41.549 INFO:teuthology.orchestra.run.smithi027.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T20:58:41.701 INFO:teuthology.orchestra.run.smithi027.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T20:58:41.710 INFO:teuthology.orchestra.run.smithi066.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T20:58:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T20:58:42.481 DEBUG:teuthology.parallel:result is None 2024-09-15T20:58:42.777 INFO:teuthology.orchestra.run.smithi027.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T20:58:43.441 INFO:teuthology.orchestra.run.smithi027.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T20:58:43.445 DEBUG:teuthology.parallel:result is None 2024-09-15T20:58:43.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:58:43.592 DEBUG:teuthology.orchestra.run.smithi027:> dpkg-query -W -f '${Version}' ceph 2024-09-15T20:58:43.611 INFO:teuthology.orchestra.run.smithi027.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.612 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.612 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T20:58:43.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:58:43.759 DEBUG:teuthology.orchestra.run.smithi066:> dpkg-query -W -f '${Version}' ceph 2024-09-15T20:58:43.779 INFO:teuthology.orchestra.run.smithi066.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.780 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.780 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T20:58:43.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T20:58:43.918 DEBUG:teuthology.orchestra.run.smithi092:> dpkg-query -W -f '${Version}' ceph 2024-09-15T20:58:43.938 INFO:teuthology.orchestra.run.smithi092.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.938 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T20:58:43.938 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T20:58:43.940 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T20:58:43.941 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:43.941 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T20:58:43.956 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:43.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T20:58:43.971 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:43.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T20:58:43.995 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T20:58:43.996 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:43.996 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T20:58:44.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T20:58:44.068 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:44.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T20:58:44.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T20:58:44.139 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:44.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T20:58:44.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T20:58:44.207 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T20:58:44.208 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:44.208 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T20:58:44.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T20:58:44.276 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:44.276 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T20:58:44.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T20:58:44.347 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:44.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T20:58:44.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T20:58:44.414 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T20:58:44.414 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:44.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T20:58:44.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T20:58:44.485 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:44.486 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T20:58:44.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T20:58:44.559 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:44.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T20:58:44.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T20:58:44.625 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T20:58:44.714 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T20:58:44.715 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2024-09-15T20:58:44.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-09-15T20:58:44.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-09-15T20:58:44.733 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T20:58:44.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T20:58:44.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T20:58:44.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T20:58:44.789 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T20:58:44.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T20:58:44.830 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T20:58:44.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T20:58:44.860 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T20:58:44.860 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300, 'osd pool default ec fast read': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T20:58:44.860 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637', 'branch': 'main', 'description': 'rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-bitmap} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7905637', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300, 'osd pool default ec fast read': True}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'], ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'], ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']], 'seed': 2855, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'sleep_before_teardown': 0, 'subset': '17936/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7498493279982dfe87f00616198a5967475f6169', 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwTUze3CAEVj9L1G7fXAlFONBA1gNoENTTzotqZuZ1kv4imoACnlmnDdd0R8aSoWyzrNAQUR1aH6Ld/dHHHQEA=', 'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFzrKEwKx2okjgEV5HtFqiO9twfbAKevzS8YKqVEmcj8VQLqhY4tt8x5jFtFAEjl6cgdfWpUMFSNwlaPZ6LaAls=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8/s6YWJ82ZJBnguSXXgjgiHXYVUOdP4LhsEpZUSj02TV/fl3m8t8N/jgYJ2+smazf6EQSptSNGuAbh2dkxJzU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300, 'osd pool default ec fast read': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 4, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_20:00:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T20:58:44.861 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T20:58:44.886 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T20:58:44.888 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T20:58:44.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T20:58:44.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T20:58:44.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T20:58:44.954 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:44.955 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T20:58:44.998 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T20:58:44.998 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 872 Links: 1 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 20:53:03.399972058 +0000 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 20:53:03.123890343 +0000 2024-09-15T20:58:45.045 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 20:53:03.123890343 +0000 2024-09-15T20:58:45.046 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-09-15T20:58:45.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T20:58:45.102 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T20:58:45.102 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T20:58:45.102 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.0004068 s, 1.3 MB/s 2024-09-15T20:58:45.103 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T20:58:45.151 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-15T20:58:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T20:58:45.197 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.197 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 893 Links: 1 2024-09-15T20:58:45.198 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.198 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 20:53:03.912123645 +0000 2024-09-15T20:58:45.198 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 20:53:03.640043114 +0000 2024-09-15T20:58:45.198 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 20:53:03.640043114 +0000 2024-09-15T20:58:45.198 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-09-15T20:58:45.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T20:58:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T20:58:45.251 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T20:58:45.252 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000381014 s, 1.3 MB/s 2024-09-15T20:58:45.253 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T20:58:45.305 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 20:53:04.396263003 +0000 2024-09-15T20:58:45.357 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 20:53:04.140189769 +0000 2024-09-15T20:58:45.358 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 20:53:04.140189769 +0000 2024-09-15T20:58:45.358 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-09-15T20:58:45.358 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T20:58:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T20:58:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T20:58:45.412 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000422514 s, 1.2 MB/s 2024-09-15T20:58:45.413 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T20:58:45.462 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-15T20:58:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T20:58:45.510 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.510 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-09-15T20:58:45.510 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.510 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-15 20:53:04.920412902 +0000 2024-09-15T20:58:45.511 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-15 20:53:04.648335092 +0000 2024-09-15T20:58:45.511 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-15 20:53:04.648335092 +0000 2024-09-15T20:58:45.511 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2024-09-15T20:58:45.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T20:58:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-15T20:58:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-15T20:58:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.0003612 s, 1.4 MB/s 2024-09-15T20:58:45.565 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T20:58:45.615 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:45.615 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:45.615 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T20:58:45.621 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T20:58:45.621 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-09-15T20:58:45.667 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-15 20:53:02.635342205 +0000 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-15 20:53:02.399344313 +0000 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-15 20:53:02.399344313 +0000 2024-09-15T20:58:45.668 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-09-15T20:58:45.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T20:58:45.725 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-15T20:58:45.725 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-15T20:58:45.725 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000444653 s, 1.2 MB/s 2024-09-15T20:58:45.726 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T20:58:45.778 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-09-15T20:58:45.828 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T20:58:45.828 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.828 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-15T20:58:45.828 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.829 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-15 20:53:03.103338025 +0000 2024-09-15T20:58:45.829 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-15 20:53:02.867340133 +0000 2024-09-15T20:58:45.829 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-15 20:53:02.867340133 +0000 2024-09-15T20:58:45.829 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-09-15T20:58:45.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T20:58:45.881 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-15T20:58:45.881 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-15T20:58:45.881 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000360518 s, 1.4 MB/s 2024-09-15T20:58:45.883 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T20:58:45.932 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-09-15T20:58:45.979 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-15 20:53:03.623333379 +0000 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-15 20:53:03.351335810 +0000 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-15 20:53:03.351335810 +0000 2024-09-15T20:58:45.980 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-09-15T20:58:45.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T20:58:46.033 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-15T20:58:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-15T20:58:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000352937 s, 1.5 MB/s 2024-09-15T20:58:46.035 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T20:58:46.085 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-15 20:53:04.083329271 +0000 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-15 20:53:03.843331414 +0000 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-15 20:53:03.843331414 +0000 2024-09-15T20:58:46.135 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: - 2024-09-15T20:58:46.136 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T20:58:46.190 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-15T20:58:46.190 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-15T20:58:46.190 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000388067 s, 1.3 MB/s 2024-09-15T20:58:46.191 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T20:58:46.244 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:46.244 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:46.244 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T20:58:46.250 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T20:58:46.250 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-15T20:58:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T20:58:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:46.299 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-15T20:58:46.300 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:46.300 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-15 20:52:59.341694701 +0000 2024-09-15T20:58:46.300 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-15 20:52:59.057697238 +0000 2024-09-15T20:58:46.300 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-15 20:52:59.057697238 +0000 2024-09-15T20:58:46.300 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-15T20:58:46.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T20:58:46.353 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-15T20:58:46.354 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-15T20:58:46.354 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000454741 s, 1.1 MB/s 2024-09-15T20:58:46.355 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T20:58:46.404 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-15T20:58:46.451 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T20:58:46.452 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:46.452 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-09-15T20:58:46.452 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:46.452 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-15 20:52:59.845690199 +0000 2024-09-15T20:58:46.453 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-15 20:52:59.589692486 +0000 2024-09-15T20:58:46.453 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-15 20:52:59.589692486 +0000 2024-09-15T20:58:46.453 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-15T20:58:46.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T20:58:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-15T20:58:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-15T20:58:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000389165 s, 1.3 MB/s 2024-09-15T20:58:46.506 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T20:58:46.555 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-15 20:53:00.309686055 +0000 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-15 20:53:00.057688306 +0000 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-15 20:53:00.057688306 +0000 2024-09-15T20:58:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-15T20:58:46.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T20:58:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-15T20:58:46.657 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-15T20:58:46.658 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000431951 s, 1.2 MB/s 2024-09-15T20:58:46.659 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T20:58:46.707 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-15T20:58:46.754 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T20:58:46.754 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-15 20:53:00.769681945 +0000 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-15 20:53:00.529684090 +0000 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-15 20:53:00.529684090 +0000 2024-09-15T20:58:46.755 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-09-15T20:58:46.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T20:58:46.808 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-15T20:58:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-15T20:58:46.809 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000347241 s, 1.5 MB/s 2024-09-15T20:58:46.810 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T20:58:46.859 INFO:tasks.ceph:osd dev map: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:46.859 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-15T20:58:46.859 INFO:tasks.ceph:Generating config... 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T20:58:46.861 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.1 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd max backfills = 2 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T20:58:46.862 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T20:58:46.863 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T20:58:46.863 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-15T20:58:46.863 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-15T20:58:46.863 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T20:58:46.863 INFO:tasks.ceph:[mon] osd pool default ec fast read = True 2024-09-15T20:58:46.864 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T20:58:46.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T20:58:46.899 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T20:58:46.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T20:58:46.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T20:58:47.044 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.66'), ('mon.c', '172.21.15.92')] 2024-09-15T20:58:47.044 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.66,172.21.15.92', 'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300, 'osd pool default ec fast read': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-15T20:58:47.045 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:47.045 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T20:58:47.093 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.66 --add c 172.21.15.92 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T20:58:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-15T20:58:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2024-09-15T20:58:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2024-09-15T20:58:47.182080+0000 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-15T20:58:47.182080+0000 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 17 (quincy) 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.b 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-15T20:58:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T20:58:47.183 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T20:58:47.229 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 07b66685-24ed-4325-b3d4-b0cceb7d7e6d... 2024-09-15T20:58:47.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T20:58:47.274 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T20:58:47.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2024-09-15T20:58:47.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.304 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T20:58:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2024-09-15T20:58:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.66,172.21.15.92" 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: mon election default strategy = 1 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd_async_recovery_min_cost = 1 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.307 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 120 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug trim objects = true 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.308 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2024-09-15T20:58:47.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: debug monc = 20 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bdev async discard = True 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bdev enable discard = True 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = bitmap 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore write v2 random = True 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore zero block detection = True 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 20 2024-09-15T20:58:47.310 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 20 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 10 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd blocked scrub grace period = 3600 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug reject backfill probability = 0.1 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify cached snaps = True 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify missing on start = True 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2024-09-15T20:58:47.311 INFO:teuthology.orchestra.run.smithi027.stdout: osd max backfills = 2 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd max markdown count = 1000 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock override recovery settings = True 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd op queue = debug_random 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd op queue cut off = debug_random 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub min interval = 60 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2024-09-15T20:58:47.312 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2024-09-15T20:58:47.313 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T20:58:47.314 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: mon scrub interval = 300 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default ec fast read = True 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2024-09-15T20:58:47.315 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2024-09-15T20:58:47.316 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T20:58:47.316 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T20:58:47.316 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2024-09-15T20:58:47.316 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2024-09-15T20:58:47.316 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-09-15T20:58:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-09-15T20:58:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.27,172.21.15.66,172.21.15.92" 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: mon election default strategy = 1 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd_async_recovery_min_cost = 1 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 120 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-09-15T20:58:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-09-15T20:58:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: debug monc = 20 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore write v2 random = True 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore zero block detection = True 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T20:58:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug reject backfill probability = 0.1 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify cached snaps = True 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify missing on start = True 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd max backfills = 2 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2024-09-15T20:58:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock override recovery settings = True 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue = debug_random 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: osd op queue cut off = debug_random 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub min interval = 60 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-09-15T20:58:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-09-15T20:58:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: mon scrub interval = 300 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default ec fast read = True 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-09-15T20:58:47.329 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.331 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.332 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-15T20:58:47.333 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.27,172.21.15.66,172.21.15.92" 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: mon election default strategy = 1 2024-09-15T20:58:47.334 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug inject dispatch delay duration = 0.1 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug inject dispatch delay probability = 0.1 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd_async_recovery_min_cost = 1 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 120 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.335 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-15T20:58:47.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: debug monc = 20 2024-09-15T20:58:47.337 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore write v2 random = True 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore zero block detection = True 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 20 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 20 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 10 2024-09-15T20:58:47.338 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd blocked scrub grace period = 3600 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug reject backfill probability = 0.1 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify cached snaps = True 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify missing on start = True 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd max backfills = 2 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd max markdown count = 1000 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock override recovery settings = True 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd op queue = debug_random 2024-09-15T20:58:47.339 INFO:teuthology.orchestra.run.smithi066.stdout: osd op queue cut off = debug_random 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub min interval = 60 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.340 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-15T20:58:47.341 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.342 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: mon scrub interval = 300 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default ec fast read = True 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-15T20:58:47.343 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T20:58:47.344 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T20:58:47.344 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-15T20:58:47.344 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-15T20:58:47.344 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-15T20:58:47.345 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T20:58:47.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T20:58:47.424 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T20:58:47.424 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:47.424 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T20:58:47.430 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:47.430 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T20:58:47.477 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:58:47.477 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:47.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T20:58:47.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T20:58:47.542 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:58:47.542 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:47.589 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:58:47.589 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:47.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T20:58:47.589 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T20:58:47.612 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:58:47.612 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:47.659 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:58:47.660 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:47.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T20:58:47.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T20:58:47.682 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:58:47.682 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:47.730 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T20:58:47.730 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T20:58:47.731 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T20:58:47.777 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T20:58:47.792 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T20:58:47.792 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T20:58:47.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T20:58:47.832 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T20:58:47.843 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T20:58:47.843 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-15T20:58:47.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T20:58:47.896 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:47.896 INFO:tasks.ceph:role: osd.0 2024-09-15T20:58:47.896 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:58:47.896 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:58:47.960 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:58:48.413 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T20:58:48.415 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T20:58:48.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T20:58:48.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T20:58:48.551 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:58:48.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:58:48.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T20:58:48.608 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:48.609 INFO:tasks.ceph:role: osd.1 2024-09-15T20:58:48.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:58:48.609 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:58:48.673 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:58:48.674 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:58:48.674 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:58:48.674 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:58:48.674 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:58:49.145 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T20:58:49.148 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T20:58:49.148 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T20:58:49.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T20:58:49.317 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:58:49.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:58:49.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T20:58:49.372 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:49.372 INFO:tasks.ceph:role: osd.2 2024-09-15T20:58:49.372 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:58:49.372 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T20:58:49.436 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:58:49.437 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:58:49.913 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T20:58:49.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T20:58:49.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T20:58:50.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T20:58:50.067 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:58:50.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:58:50.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T20:58:50.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T20:58:50.123 INFO:tasks.ceph:role: osd.3 2024-09-15T20:58:50.123 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T20:58:50.123 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T20:58:50.183 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:58:50.184 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:58:50.611 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-09-15T20:58:50.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-09-15T20:58:50.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T20:58:50.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T20:58:50.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:58:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:58:50.722 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:50.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:50.942+0000 7f9ff0f517c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T20:58:50.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:50.942+0000 7f9ff0f517c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T20:58:50.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:50.942+0000 7f9ff0f517c0 -1 bdev(0x56523e0e3800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T20:58:50.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:50.942+0000 7f9ff0f517c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T20:58:54.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T20:58:54.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:54.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:54.457+0000 7fb1542037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T20:58:54.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:54.457+0000 7fb1542037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T20:58:54.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:54.457+0000 7fb1542037c0 -1 bdev(0x55b84b327800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T20:58:54.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:54.457+0000 7fb1542037c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T20:58:57.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T20:58:57.653 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:58:57.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:57.733+0000 7f45582ed7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T20:58:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:57.733+0000 7f45582ed7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T20:58:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:57.733+0000 7f45582ed7c0 -1 bdev(0x55d8b0a97800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T20:58:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:58:57.737+0000 7f45582ed7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T20:59:00.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T20:59:00.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:01.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:59:01.085+0000 7fa86bdb47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T20:59:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:59:01.085+0000 7fa86bdb47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T20:59:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:59:01.085+0000 7fa86bdb47c0 -1 bdev(0x555aaff43800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:01.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-15T20:59:01.085+0000 7fa86bdb47c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T20:59:04.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T20:59:04.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T20:59:04.250 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:04.250 INFO:tasks.ceph:role: osd.4 2024-09-15T20:59:04.250 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:59:04.250 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:04.315 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:04.316 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:04.316 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:04.316 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:04.316 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:04.770 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-15T20:59:04.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-15T20:59:04.772 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T20:59:04.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T20:59:04.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:04.875 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:04.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T20:59:04.929 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:04.929 INFO:tasks.ceph:role: osd.5 2024-09-15T20:59:04.929 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:59:04.929 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:04.991 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:04.992 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:04.992 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:04.992 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:04.992 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:05.448 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-15T20:59:05.450 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-15T20:59:05.450 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T20:59:05.595 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T20:59:05.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:05.607 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:05.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T20:59:05.663 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:05.663 INFO:tasks.ceph:role: osd.6 2024-09-15T20:59:05.663 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:59:05.663 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:05.727 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:05.728 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:05.728 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:05.728 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:06.211 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-15T20:59:06.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-15T20:59:06.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T20:59:06.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T20:59:06.373 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:06.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:06.374 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T20:59:06.432 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:06.432 INFO:tasks.ceph:role: osd.7 2024-09-15T20:59:06.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T20:59:06.432 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:06.494 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:06.495 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:06.495 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:06.928 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-15T20:59:06.930 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-15T20:59:06.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T20:59:07.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T20:59:07.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:07.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:07.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:07.144+0000 7f856ece67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T20:59:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:07.148+0000 7f856ece67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T20:59:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:07.148+0000 7f856ece67c0 -1 bdev(0x564b16605800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:07.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:07.148+0000 7f856ece67c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T20:59:10.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T20:59:10.323 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:10.404+0000 7f8c441c57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T20:59:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:10.404+0000 7f8c441c57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T20:59:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:10.404+0000 7f8c441c57c0 -1 bdev(0x55c031143800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:10.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:10.404+0000 7f8c441c57c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T20:59:13.556 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T20:59:13.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:13.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:13.692+0000 7f19190857c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T20:59:13.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:13.692+0000 7f19190857c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T20:59:13.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:13.692+0000 7f19190857c0 -1 bdev(0x5602f540d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:13.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:13.692+0000 7f19190857c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T20:59:16.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T20:59:16.888 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:16.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:16.968+0000 7fcf5ebd47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T20:59:16.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:16.968+0000 7fcf5ebd47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T20:59:16.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:16.968+0000 7fcf5ebd47c0 -1 bdev(0x55eab7f83800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:16.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-15T20:59:16.968+0000 7fcf5ebd47c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T20:59:20.101 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T20:59:20.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-15T20:59:20.170 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:20.171 INFO:tasks.ceph:role: osd.8 2024-09-15T20:59:20.171 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:59:20.171 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T20:59:20.233 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:20.234 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:20.235 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:20.235 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:20.688 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-15T20:59:20.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-15T20:59:20.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-8 2024-09-15T20:59:20.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-15T20:59:20.832 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:20.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-15T20:59:20.890 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:20.890 INFO:tasks.ceph:role: osd.9 2024-09-15T20:59:20.890 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:59:20.890 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T20:59:20.954 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:20.954 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:20.954 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:20.954 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:20.954 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:20.955 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:20.955 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:20.955 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:20.955 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:20.955 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:21.417 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-15T20:59:21.419 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-15T20:59:21.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-9 2024-09-15T20:59:21.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-15T20:59:21.593 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:21.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:21.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-15T20:59:21.649 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:21.649 INFO:tasks.ceph:role: osd.10 2024-09-15T20:59:21.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:59:21.650 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T20:59:21.714 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:21.714 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:21.715 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:22.194 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-15T20:59:22.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-15T20:59:22.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-15T20:59:22.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-15T20:59:22.359 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:22.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:22.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-15T20:59:22.413 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-15T20:59:22.413 INFO:tasks.ceph:role: osd.11 2024-09-15T20:59:22.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T20:59:22.413 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T20:59:22.477 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T20:59:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T20:59:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T20:59:22.478 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T20:59:22.903 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-09-15T20:59:22.904 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-09-15T20:59:22.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-11 2024-09-15T20:59:22.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-15T20:59:23.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T20:59:23.044 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T20:59:23.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:23.147+0000 7fabb370f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-15T20:59:23.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:23.151+0000 7fabb370f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-15T20:59:23.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:23.151+0000 7fabb370f7c0 -1 bdev(0x55b03a443800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:23.155 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:23.151+0000 7fabb370f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-15T20:59:26.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-15T20:59:26.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:26.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:26.395+0000 7fbcc37c67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-15T20:59:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:26.395+0000 7fbcc37c67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-15T20:59:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:26.395+0000 7fbcc37c67c0 -1 bdev(0x558066fab800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:26.395+0000 7fbcc37c67c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-15T20:59:29.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-15T20:59:29.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:29.672 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:29.667+0000 7ff3763fd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-15T20:59:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:29.671+0000 7ff3763fd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-15T20:59:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:29.671+0000 7ff3763fd7c0 -1 bdev(0x5590c1329800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:29.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:29.671+0000 7ff3763fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-15T20:59:32.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-15T20:59:32.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:32.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:32.971+0000 7fdba1a1a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-15T20:59:32.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:32.971+0000 7fdba1a1a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-15T20:59:32.975 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:32.971+0000 7fdba1a1a7c0 -1 bdev(0x55db708e5800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2024-09-15T20:59:32.976 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-15T20:59:32.971+0000 7fdba1a1a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-15T20:59:36.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-15T20:59:36.154 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T20:59:36.154 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:36.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T20:59:36.169 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:36.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T20:59:36.224 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:36.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T20:59:36.279 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:36.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T20:59:36.337 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:36.337 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T20:59:36.353 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:36.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T20:59:36.407 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:36.407 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T20:59:36.463 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:36.463 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T20:59:36.519 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:36.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T20:59:36.574 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:36.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-15T20:59:36.588 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:36.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-15T20:59:36.644 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:36.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-15T20:59:36.705 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:36.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-15T20:59:36.761 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:36.761 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T20:59:36.767 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T20:59:36.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T20:59:36.813 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T20:59:36.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCKSudmywA3OBAAUY3FfzSQd19UBWOXcJvHPg== 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCOSudmr+d9GxAAppC5tNOuCPyuZLtrp9Da+w== 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCRSudmG7v1KxAAubynAQ3osoJvjX+0osLIDg== 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2024-09-15T20:59:36.823 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCVSudmUUkqBRAAusKelaRWGodoNbVb5mjLLw== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCHSudm0oBKLhAAJ//4ZzgwD4ZEiMuuZ04Klw== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCbSudmpY3fCBAAs/0CBEj7sykuhvabDNZOGQ== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCeSudmvVc6GBAAcqjFjSd7iCbBjW4L4n5A7g== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQChSudm1LlqKRAAJI/ntFEcxmXlf1e5laXI8w== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCkSudmQj7aORAAZatjFQmhaWjGgv24pURtNg== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.8] 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCrSudmnV4ZCRAAVkIGCUhzuXznJ34IIYr2jQ== 2024-09-15T20:59:36.824 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.9] 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCuSudmHT/BFxAAWRMGHKVzjlOVQ1P5mnR7dw== 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.10] 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCxSudmoQcvKBAAiucE83SJube1FxcOa8fqFg== 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.11] 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQC0SudmD/80OhAA9I3AyZlOLWic1T+XSQZGbw== 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCHSudmKMCyMRAAQtoYZ+YXJbOn9fnSTPe0nA== 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-09-15T20:59:36.825 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCKSudmywA3OBAAUY3FfzSQd19UBWOXcJvHPg== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCOSudmr+d9GxAAppC5tNOuCPyuZLtrp9Da+w== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCRSudmG7v1KxAAubynAQ3osoJvjX+0osLIDg== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCVSudmUUkqBRAAusKelaRWGodoNbVb5mjLLw== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCHSudm0oBKLhAAJ//4ZzgwD4ZEiMuuZ04Klw== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCbSudmpY3fCBAAs/0CBEj7sykuhvabDNZOGQ== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCeSudmvVc6GBAAcqjFjSd7iCbBjW4L4n5A7g== 2024-09-15T20:59:36.826 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQChSudm1LlqKRAAJI/ntFEcxmXlf1e5laXI8w== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCkSudmQj7aORAAZatjFQmhaWjGgv24pURtNg== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.8] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCrSudmnV4ZCRAAVkIGCUhzuXznJ34IIYr2jQ== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.9] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCuSudmHT/BFxAAWRMGHKVzjlOVQ1P5mnR7dw== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.10] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCxSudmoQcvKBAAiucE83SJube1FxcOa8fqFg== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.11] 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQC0SudmD/80OhAA9I3AyZlOLWic1T+XSQZGbw== 2024-09-15T20:59:36.827 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-15T20:59:36.828 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCHSudmKMCyMRAAQtoYZ+YXJbOn9fnSTPe0nA== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCKSudmywA3OBAAUY3FfzSQd19UBWOXcJvHPg== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCOSudmr+d9GxAAppC5tNOuCPyuZLtrp9Da+w== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCRSudmG7v1KxAAubynAQ3osoJvjX+0osLIDg== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCVSudmUUkqBRAAusKelaRWGodoNbVb5mjLLw== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCHSudm0oBKLhAAJ//4ZzgwD4ZEiMuuZ04Klw== 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-09-15T20:59:36.830 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCbSudmpY3fCBAAs/0CBEj7sykuhvabDNZOGQ== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeSudmvVc6GBAAcqjFjSd7iCbBjW4L4n5A7g== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQChSudm1LlqKRAAJI/ntFEcxmXlf1e5laXI8w== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCkSudmQj7aORAAZatjFQmhaWjGgv24pURtNg== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.8] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrSudmnV4ZCRAAVkIGCUhzuXznJ34IIYr2jQ== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.9] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCuSudmHT/BFxAAWRMGHKVzjlOVQ1P5mnR7dw== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.10] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCxSudmoQcvKBAAiucE83SJube1FxcOa8fqFg== 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.11] 2024-09-15T20:59:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC0SudmD/80OhAA9I3AyZlOLWic1T+XSQZGbw== 2024-09-15T20:59:36.832 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-09-15T20:59:36.832 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCHSudmKMCyMRAAQtoYZ+YXJbOn9fnSTPe0nA== 2024-09-15T20:59:36.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:36.873 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:36.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:36.922 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:36.924 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:36.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.009 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T20:59:37.181 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T20:59:37.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T20:59:37.270 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.421 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.585 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.671 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.757 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T20:59:37.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T20:59:37.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T20:59:37.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T20:59:38.020 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T20:59:38.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T20:59:38.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T20:59:38.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T20:59:38.139 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T20:59:38.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T20:59:38.247 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T20:59:38.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T20:59:38.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T20:59:38.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T20:59:38.379 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:38.381 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:38.384 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T20:59:38.425 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T20:59:38.425 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T20:59:38.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T20:59:38.429 INFO:tasks.ceph.mon.a:Started 2024-09-15T20:59:38.429 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T20:59:38.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T20:59:38.432 INFO:tasks.ceph.mon.b:Started 2024-09-15T20:59:38.432 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T20:59:38.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T20:59:38.470 INFO:tasks.ceph.mon.c:Started 2024-09-15T20:59:38.470 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T20:59:38.470 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T20:59:38.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T20:59:38.473 INFO:tasks.ceph.mgr.x:Started 2024-09-15T20:59:38.473 DEBUG:tasks.ceph:set 0 configs 2024-09-15T20:59:38.473 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T20:59:38.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T20:59:43.995 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-15T20:59:44.015 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T20:59:44.015 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T20:59:44.015 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:44.015 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T20:59:44.028 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:44.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T20:59:44.082 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:44.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T20:59:44.149 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-15T20:59:44.149 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T20:59:44.209 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:44.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T20:59:44.228 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:44.228 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T20:59:44.292 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:44.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T20:59:44.360 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-15T20:59:44.361 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T20:59:44.423 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:44.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-15T20:59:44.439 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:44.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-15T20:59:44.508 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:44.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-15T20:59:44.567 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-15T20:59:44.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-15T20:59:44.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd 0 2024-09-15T20:59:44.971 INFO:teuthology.orchestra.run.smithi092.stdout:0 2024-09-15T20:59:44.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 43960b2a-b250-45e3-b725-484c0751590d 1 2024-09-15T20:59:45.197 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-09-15T20:59:45.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 9653f0df-bf71-4a94-b79a-498f067c8de8 2 2024-09-15T20:59:45.421 INFO:teuthology.orchestra.run.smithi092.stdout:2 2024-09-15T20:59:45.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 9ac2adf5-f248-4838-9bd5-57d13dd53309 3 2024-09-15T20:59:45.645 INFO:teuthology.orchestra.run.smithi092.stdout:3 2024-09-15T20:59:45.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 72c3acd6-6e64-4853-8823-947f944b96b5 4 2024-09-15T20:59:45.872 INFO:teuthology.orchestra.run.smithi092.stdout:4 2024-09-15T20:59:45.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new ad64f860-44e7-45a6-bc0d-e85458d80f39 5 2024-09-15T20:59:46.101 INFO:teuthology.orchestra.run.smithi092.stdout:5 2024-09-15T20:59:46.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 4701a955-3643-4210-800d-7a86ba5155bc 6 2024-09-15T20:59:46.323 INFO:teuthology.orchestra.run.smithi092.stdout:6 2024-09-15T20:59:46.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new abc724fc-25c2-4b2e-b4ba-16b61dbaf89d 7 2024-09-15T20:59:46.538 INFO:teuthology.orchestra.run.smithi092.stdout:7 2024-09-15T20:59:46.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new b114589d-5020-4c1c-ad76-f07231e58d47 8 2024-09-15T20:59:46.762 INFO:teuthology.orchestra.run.smithi092.stdout:8 2024-09-15T20:59:46.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 6e4c938a-4505-41f5-90dc-416c6ceb868a 9 2024-09-15T20:59:46.980 INFO:teuthology.orchestra.run.smithi092.stdout:9 2024-09-15T20:59:46.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new fa725c67-48fe-45ac-bcfb-6cf3fddea7fe 10 2024-09-15T20:59:47.188 INFO:teuthology.orchestra.run.smithi092.stdout:10 2024-09-15T20:59:47.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new c6fcefe9-5105-45bf-ac2d-ca8eb186316c 11 2024-09-15T20:59:47.398 INFO:teuthology.orchestra.run.smithi092.stdout:11 2024-09-15T20:59:47.415 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T20:59:47.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T20:59:47.418 INFO:tasks.ceph.osd.0:Started 2024-09-15T20:59:47.418 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T20:59:47.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T20:59:47.420 INFO:tasks.ceph.osd.1:Started 2024-09-15T20:59:47.420 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T20:59:47.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T20:59:47.423 INFO:tasks.ceph.osd.2:Started 2024-09-15T20:59:47.423 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T20:59:47.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T20:59:47.425 INFO:tasks.ceph.osd.3:Started 2024-09-15T20:59:47.426 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T20:59:47.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T20:59:47.427 INFO:tasks.ceph.mgr.x.smithi066.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-15T20:59:47.427 INFO:tasks.ceph.mgr.x.smithi066.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-15T20:59:47.427 INFO:tasks.ceph.mgr.x.smithi066.stderr: from numpy import show_config as show_numpy_config 2024-09-15T20:59:47.428 INFO:tasks.ceph.osd.4:Started 2024-09-15T20:59:47.428 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T20:59:47.428 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T20:59:47.431 INFO:tasks.ceph.osd.5:Started 2024-09-15T20:59:47.431 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T20:59:47.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T20:59:47.433 INFO:tasks.ceph.osd.6:Started 2024-09-15T20:59:47.433 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T20:59:47.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T20:59:47.436 INFO:tasks.ceph.osd.7:Started 2024-09-15T20:59:47.436 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-15T20:59:47.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-15T20:59:47.438 INFO:tasks.ceph.osd.8:Started 2024-09-15T20:59:47.438 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-15T20:59:47.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-15T20:59:47.440 INFO:tasks.ceph.osd.9:Started 2024-09-15T20:59:47.440 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-15T20:59:47.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-15T20:59:47.442 INFO:tasks.ceph.osd.10:Started 2024-09-15T20:59:47.442 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-15T20:59:47.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-15T20:59:47.445 INFO:tasks.ceph.osd.11:Started 2024-09-15T20:59:47.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T20:59:47.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T20:59:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":14,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T20:59:47.394909+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T20:59:47.670 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T20:59:47.670 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T20:59:48.511 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T20:59:48.508+0000 7f5b126607c0 -1 Falling back to public interface 2024-09-15T20:59:48.609 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T20:59:48.609+0000 7fbc25b647c0 -1 Falling back to public interface 2024-09-15T20:59:48.666 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T20:59:48.663+0000 7fc95e6ac7c0 -1 Falling back to public interface 2024-09-15T20:59:48.740 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T20:59:48.736+0000 7fd55a83f7c0 -1 Falling back to public interface 2024-09-15T20:59:48.837 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T20:59:48.837+0000 7fe659c967c0 -1 Falling back to public interface 2024-09-15T20:59:48.876 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T20:59:48.877+0000 7f8879ce77c0 -1 Falling back to public interface 2024-09-15T20:59:48.880 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T20:59:48.890 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T20:59:48.887+0000 7f86763b77c0 -1 Falling back to public interface 2024-09-15T20:59:48.902 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T20:59:48.899+0000 7f84dcf6c7c0 -1 Falling back to public interface 2024-09-15T20:59:49.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T20:59:49.024+0000 7fde98b757c0 -1 Falling back to public interface 2024-09-15T20:59:49.072 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T20:59:49.072 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":14,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T20:59:47.394909+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T20:59:49.088 DEBUG:teuthology.misc:0 of 12 OSDs are up 2024-09-15T20:59:49.281 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T20:59:49.281+0000 7ff6e1c417c0 -1 Falling back to public interface 2024-09-15T20:59:49.312 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T20:59:49.308+0000 7f61efda07c0 -1 Falling back to public interface 2024-09-15T20:59:49.360 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T20:59:49.355+0000 7f5a95c1c7c0 -1 Falling back to public interface 2024-09-15T20:59:50.386 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-09-15T20:59:50.380+0000 7fd9e2db8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T20:59:50.386 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-09-15T20:59:50.380+0000 7fd9e2db8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T20:59:52.624 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T20:59:52.620+0000 7f5b126607c0 -1 osd.4 0 log_to_monitors true 2024-09-15T20:59:52.657 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T20:59:52.655+0000 7fc95e6ac7c0 -1 osd.9 0 log_to_monitors true 2024-09-15T20:59:52.667 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T20:59:52.669+0000 7fbc25b647c0 -1 osd.1 0 log_to_monitors true 2024-09-15T20:59:52.774 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T20:59:52.773+0000 7fe659c967c0 -1 osd.0 0 log_to_monitors true 2024-09-15T20:59:52.791 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T20:59:52.788+0000 7fd55a83f7c0 -1 osd.6 0 log_to_monitors true 2024-09-15T20:59:52.884 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T20:59:52.883+0000 7f84dcf6c7c0 -1 osd.11 0 log_to_monitors true 2024-09-15T20:59:52.932 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T20:59:52.933+0000 7f8879ce77c0 -1 osd.3 0 log_to_monitors true 2024-09-15T20:59:52.980 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T20:59:52.975+0000 7f86763b77c0 -1 osd.10 0 log_to_monitors true 2024-09-15T20:59:53.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T20:59:53.148+0000 7fde98b757c0 -1 osd.5 0 log_to_monitors true 2024-09-15T20:59:53.417 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T20:59:53.417+0000 7ff6e1c417c0 -1 osd.2 0 log_to_monitors true 2024-09-15T20:59:53.577 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T20:59:53.575+0000 7f5a95c1c7c0 -1 osd.8 0 log_to_monitors true 2024-09-15T20:59:53.653 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T20:59:53.648+0000 7f61efda07c0 -1 osd.7 0 log_to_monitors true 2024-09-15T20:59:54.404 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T20:59:54.405+0000 7fbc21b0d640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T20:59:54.404 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T20:59:54.405+0000 7f8875c90640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T20:59:54.405 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T20:59:54.405+0000 7fe655c3f640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T20:59:54.406 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T20:59:54.403+0000 7fc95a655640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T20:59:54.408 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T20:59:54.404+0000 7f5b0e609640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T20:59:54.409 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T20:59:54.404+0000 7fd5567e8640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T20:59:54.416 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T20:59:54.417+0000 7fbc1d947640 -1 osd.1 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:54.423 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T20:59:54.420+0000 7fd552622640 -1 osd.6 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:54.424 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T20:59:54.425+0000 7f8871aca640 -1 osd.3 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:54.428 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T20:59:54.429+0000 7fe651a79640 -1 osd.0 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:54.451 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T20:59:54.448+0000 7f5b0a443640 -1 osd.4 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:54.500 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T20:59:54.495+0000 7fc95648f640 -1 osd.9 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:55.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T20:59:55.212+0000 7fde95330640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T20:59:55.243 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T20:59:55.240+0000 7fde8c950640 -1 osd.5 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:55.499 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T20:59:55.495+0000 7f5a91bc5640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T20:59:55.506 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T20:59:55.500+0000 7f61ebd49640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T20:59:55.528 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T20:59:55.524+0000 7f61e7b83640 -1 osd.7 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:55.529 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T20:59:55.527+0000 7f5a8d9ff640 -1 osd.8 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:55.587 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T20:59:55.583+0000 7f8672360640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T20:59:55.625 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T20:59:55.623+0000 7f866a192640 -1 osd.10 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:55.660 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T20:59:55.655+0000 7f84d9727640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T20:59:55.677 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T20:59:55.675+0000 7f84d0d47640 -1 osd.11 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:56.291 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T20:59:56.289+0000 7ff6ddbea640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T20:59:56.299 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T20:59:56.412 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T20:59:56.413+0000 7ff6d5a1c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T20:59:56.600 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T20:59:56.601 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":18,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T20:59:56.411296+0000","last_up_change":"2024-09-15T20:59:56.411296+0000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.788074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.674356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.966270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.604135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.110212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.787136+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.645022+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T20:59:56.617 DEBUG:teuthology.misc:11 of 12 OSDs are up 2024-09-15T21:00:03.828 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:00:04.144 INFO:teuthology.misc.health.smithi027.stdout: 2024-09-15T21:00:04.144 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":20,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T20:59:58.429345+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.788074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.674356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.389412+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.966270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.604135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.110212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.787136+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.668403+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.568433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.645022+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.965253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.847328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:00:04.162 DEBUG:teuthology.misc:12 of 12 OSDs are up 2024-09-15T21:00:04.162 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:00:04.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:00:04.871 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'rbd' created 2024-09-15T21:00:04.897 DEBUG:teuthology.orchestra.run.smithi027:> rbd --cluster ceph pool init rbd 2024-09-15T21:00:08.020 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:00:08.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T21:00:08.022 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:00:08.341 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:08.360 INFO:teuthology.orchestra.run.smithi027.stdout:[{"version":5,"timestamp":"2024-09-15T20:59:56.294452+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"10252.227383"}]}] 2024-09-15T21:00:08.361 INFO:tasks.ceph_manager:config epoch is 5 2024-09-15T21:00:08.361 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:00:08.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:00:08.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:00:08.721 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:08.737 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":872884401},{"type":"v1","addr":"172.21.15.66:6833","nonce":872884401}]},"active_addr":"172.21.15.66:6833/872884401","active_change":"2024-09-15T20:59:49.364821+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3817771550}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2172932773}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1305596904}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":138877430}]}]} 2024-09-15T21:00:08.739 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:00:08.739 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:00:08.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:00:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:09.036 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:00:07.891435+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.788074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.674356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.389412+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.966270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.604135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.110212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.787136+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.668403+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.568433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.645022+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.965253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.847328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:00:09.053 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:00:09.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:00:09.358 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:09.358 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:00:07.891435+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T20:59:47.394909+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.788074+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.674356+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.389412+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.966270+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.604135+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.110212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.787136+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.668403+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:54.568433+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.645022+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.965253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T20:59:53.847328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:00:09.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:00:09.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:00:09.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:00:09.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:00:09.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:00:09.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:00:09.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:00:09.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:00:09.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T21:00:09.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T21:00:09.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T21:00:09.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T21:00:09.749 INFO:teuthology.orchestra.run.smithi027.stdout:77309411332 2024-09-15T21:00:09.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T21:00:09.751 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:09.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:00:09.756 INFO:teuthology.orchestra.run.smithi027.stdout:73014444038 2024-09-15T21:00:09.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:00:09.757 INFO:teuthology.orchestra.run.smithi027.stdout:77309411333 2024-09-15T21:00:09.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T21:00:09.770 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:09.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:00:09.772 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:09.773 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T21:00:09.783 INFO:teuthology.orchestra.run.smithi027.stdout:77309411334 2024-09-15T21:00:09.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T21:00:09.786 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:09.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:00:09.796 INFO:teuthology.orchestra.run.smithi027.stdout:81604378630 2024-09-15T21:00:09.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:00:09.798 INFO:teuthology.orchestra.run.smithi027.stdout:73014444038 2024-09-15T21:00:09.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:00:09.802 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:09.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:00:09.826 INFO:teuthology.orchestra.run.smithi027.stdout:77309411332 2024-09-15T21:00:09.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:00:10.415 INFO:teuthology.orchestra.run.smithi027.stdout:73014444035 2024-09-15T21:00:10.439 INFO:teuthology.orchestra.run.smithi027.stdout:77309411332 2024-09-15T21:00:10.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.4 2024-09-15T21:00:10.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411332 for osd.10 2024-09-15T21:00:10.529 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:10.536 INFO:teuthology.orchestra.run.smithi027.stdout:77309411331 2024-09-15T21:00:10.540 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:10.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.8 2024-09-15T21:00:10.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444037 for osd.3 2024-09-15T21:00:10.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444037 for osd.6 2024-09-15T21:00:10.601 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:10.614 INFO:teuthology.orchestra.run.smithi027.stdout:73014444035 2024-09-15T21:00:10.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444036 for osd.0 2024-09-15T21:00:10.635 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:10.635 INFO:teuthology.orchestra.run.smithi027.stdout:81604378629 2024-09-15T21:00:10.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444035 for osd.1 2024-09-15T21:00:10.650 INFO:teuthology.orchestra.run.smithi027.stdout:77309411331 2024-09-15T21:00:10.654 INFO:teuthology.orchestra.run.smithi027.stdout:77309411333 2024-09-15T21:00:10.654 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:10.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444036 for osd.9 2024-09-15T21:00:10.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378630 got 81604378629 for osd.2 2024-09-15T21:00:10.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411334 got 77309411333 for osd.11 2024-09-15T21:00:10.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.7 2024-09-15T21:00:10.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444036 for osd.5 2024-09-15T21:00:11.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:00:11.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T21:00:11.565 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T21:00:11.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:00:11.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:00:11.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:00:11.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:00:11.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T21:00:11.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:00:11.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T21:00:11.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:00:11.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:00:11.960 INFO:teuthology.orchestra.run.smithi027.stdout:73014444036 2024-09-15T21:00:12.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.4 2024-09-15T21:00:12.006 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.115 INFO:teuthology.orchestra.run.smithi027.stdout:77309411333 2024-09-15T21:00:12.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.10 2024-09-15T21:00:12.147 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.179 INFO:teuthology.orchestra.run.smithi027.stdout:77309411332 2024-09-15T21:00:12.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411332 for osd.8 2024-09-15T21:00:12.243 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.305 INFO:teuthology.orchestra.run.smithi027.stdout:73014444038 2024-09-15T21:00:12.310 INFO:teuthology.orchestra.run.smithi027.stdout:73014444038 2024-09-15T21:00:12.316 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:12.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444038 for osd.6 2024-09-15T21:00:12.344 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:12.345 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444037 for osd.1 2024-09-15T21:00:12.348 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.357 INFO:teuthology.orchestra.run.smithi027.stdout:73014444038 2024-09-15T21:00:12.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444038 got 73014444038 for osd.3 2024-09-15T21:00:12.358 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444037 for osd.9 2024-09-15T21:00:12.373 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444038 for osd.0 2024-09-15T21:00:12.385 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.413 INFO:teuthology.orchestra.run.smithi027.stdout:77309411334 2024-09-15T21:00:12.429 INFO:teuthology.orchestra.run.smithi027.stdout:77309411332 2024-09-15T21:00:12.431 INFO:teuthology.orchestra.run.smithi027.stdout:81604378630 2024-09-15T21:00:12.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411334 got 77309411334 for osd.11 2024-09-15T21:00:12.438 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411332 for osd.7 2024-09-15T21:00:12.449 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378630 got 81604378630 for osd.2 2024-09-15T21:00:12.451 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.454 INFO:teuthology.orchestra.run.smithi027.stdout:73014444037 2024-09-15T21:00:12.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444037 got 73014444037 for osd.5 2024-09-15T21:00:12.471 DEBUG:teuthology.parallel:result is None 2024-09-15T21:00:12.471 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:00:12.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:12.802 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:12.818 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T21:00:11.376193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328220,"kb_used_data":3892,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133860,"statfs":{"total":1159641169920,"available":1159305072640,"internally_reserved":0,"allocated":3985408,"data_stored":2258254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":331855163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":60,"num_write_kb":1308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001879"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.463621+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:08.463621+0000","last_peered":"2024-09-15T21:00:08.463621+0000","last_clean":"2024-09-15T21:00:08.463621+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:08.463621+0000","last_undegraded":"2024-09-15T21:00:08.463621+0000","last_fullsized":"2024-09-15T21:00:08.463621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902562+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:07.902562+0000","last_peered":"2024-09-15T21:00:07.902562+0000","last_clean":"2024-09-15T21:00:07.902562+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:07.902562+0000","last_undegraded":"2024-09-15T21:00:07.902562+0000","last_fullsized":"2024-09-15T21:00:07.902562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.265411+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:08.265411+0000","last_peered":"2024-09-15T21:00:08.265411+0000","last_clean":"2024-09-15T21:00:08.265411+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:08.265411+0000","last_undegraded":"2024-09-15T21:00:08.265411+0000","last_fullsized":"2024-09-15T21:00:08.265411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902447+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:07.902447+0000","last_peered":"2024-09-15T21:00:07.902447+0000","last_clean":"2024-09-15T21:00:07.902447+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:07.902447+0000","last_undegraded":"2024-09-15T21:00:07.902447+0000","last_fullsized":"2024-09-15T21:00:07.902447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.906899+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:07.906899+0000","last_peered":"2024-09-15T21:00:07.906899+0000","last_clean":"2024-09-15T21:00:07.906899+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:07.906899+0000","last_undegraded":"2024-09-15T21:00:07.906899+0000","last_fullsized":"2024-09-15T21:00:07.906899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.928974+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:07.928974+0000","last_peered":"2024-09-15T21:00:07.928974+0000","last_clean":"2024-09-15T21:00:07.928974+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:07.928974+0000","last_undegraded":"2024-09-15T21:00:07.928974+0000","last_fullsized":"2024-09-15T21:00:07.928974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.262014+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:08.262014+0000","last_peered":"2024-09-15T21:00:08.262014+0000","last_clean":"2024-09-15T21:00:08.262014+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:08.262014+0000","last_undegraded":"2024-09-15T21:00:08.262014+0000","last_fullsized":"2024-09-15T21:00:08.262014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.905273+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:07.905273+0000","last_peered":"2024-09-15T21:00:07.905273+0000","last_clean":"2024-09-15T21:00:07.905273+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:07.905273+0000","last_undegraded":"2024-09-15T21:00:07.905273+0000","last_fullsized":"2024-09-15T21:00:07.905273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":134,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.361670+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:08.361670+0000","last_peered":"2024-09-15T21:00:08.361670+0000","last_clean":"2024-09-15T21:00:08.361670+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:08.361670+0000","last_undegraded":"2024-09-15T21:00:08.361670+0000","last_fullsized":"2024-09-15T21:00:08.361670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411334,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":623838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":685278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:12.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:13.118 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:13.118 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:13.135 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-15T21:00:11.376193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328220,"kb_used_data":3892,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133860,"statfs":{"total":1159641169920,"available":1159305072640,"internally_reserved":0,"allocated":3985408,"data_stored":2258254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":331855163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":27,"num_write":60,"num_write_kb":1308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001879"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.463621+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:08.463621+0000","last_peered":"2024-09-15T21:00:08.463621+0000","last_clean":"2024-09-15T21:00:08.463621+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:08.463621+0000","last_undegraded":"2024-09-15T21:00:08.463621+0000","last_fullsized":"2024-09-15T21:00:08.463621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902562+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:07.902562+0000","last_peered":"2024-09-15T21:00:07.902562+0000","last_clean":"2024-09-15T21:00:07.902562+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:07.902562+0000","last_undegraded":"2024-09-15T21:00:07.902562+0000","last_fullsized":"2024-09-15T21:00:07.902562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.265411+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:08.265411+0000","last_peered":"2024-09-15T21:00:08.265411+0000","last_clean":"2024-09-15T21:00:08.265411+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:08.265411+0000","last_undegraded":"2024-09-15T21:00:08.265411+0000","last_fullsized":"2024-09-15T21:00:08.265411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902447+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:07.902447+0000","last_peered":"2024-09-15T21:00:07.902447+0000","last_clean":"2024-09-15T21:00:07.902447+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:07.902447+0000","last_undegraded":"2024-09-15T21:00:07.902447+0000","last_fullsized":"2024-09-15T21:00:07.902447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.906899+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:07.906899+0000","last_peered":"2024-09-15T21:00:07.906899+0000","last_clean":"2024-09-15T21:00:07.906899+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:07.906899+0000","last_undegraded":"2024-09-15T21:00:07.906899+0000","last_fullsized":"2024-09-15T21:00:07.906899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.928974+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:07.928974+0000","last_peered":"2024-09-15T21:00:07.928974+0000","last_clean":"2024-09-15T21:00:07.928974+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:07.928974+0000","last_undegraded":"2024-09-15T21:00:07.928974+0000","last_fullsized":"2024-09-15T21:00:07.928974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.262014+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:08.262014+0000","last_peered":"2024-09-15T21:00:08.262014+0000","last_clean":"2024-09-15T21:00:08.262014+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:08.262014+0000","last_undegraded":"2024-09-15T21:00:08.262014+0000","last_fullsized":"2024-09-15T21:00:08.262014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.905273+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:07.905273+0000","last_peered":"2024-09-15T21:00:07.905273+0000","last_clean":"2024-09-15T21:00:07.905273+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:07.905273+0000","last_undegraded":"2024-09-15T21:00:07.905273+0000","last_fullsized":"2024-09-15T21:00:07.905273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":134,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.361670+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:08.361670+0000","last_peered":"2024-09-15T21:00:08.361670+0000","last_clean":"2024-09-15T21:00:08.361670+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:08.361670+0000","last_undegraded":"2024-09-15T21:00:08.361670+0000","last_fullsized":"2024-09-15T21:00:08.361670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378630,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411334,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":623838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":685278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444037,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:13.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:00:13.136 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:00:13.136 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:00:13.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:00:13.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:13.470 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:00:13.488 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:00:13.488 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T21:00:13.499 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 4, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T21:00:13.499 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300, 'osd pool default ec fast read': True}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}} 2024-09-15T21:00:13.499 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 4, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T21:00:13.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T21:00:13.827 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T21:00:13.827 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:00:13.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:14.115 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:14.115 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:14.132 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-15T21:00:13.523613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328220,"kb_used_data":3892,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133860,"statfs":{"total":1159641169920,"available":1159305072640,"internally_reserved":0,"allocated":3985408,"data_stored":2258254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":331855163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.641169"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.463621+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:08.463621+0000","last_peered":"2024-09-15T21:00:08.463621+0000","last_clean":"2024-09-15T21:00:08.463621+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:08.463621+0000","last_undegraded":"2024-09-15T21:00:08.463621+0000","last_fullsized":"2024-09-15T21:00:08.463621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902562+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:07.902562+0000","last_peered":"2024-09-15T21:00:07.902562+0000","last_clean":"2024-09-15T21:00:07.902562+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:07.902562+0000","last_undegraded":"2024-09-15T21:00:07.902562+0000","last_fullsized":"2024-09-15T21:00:07.902562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.265411+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:08.265411+0000","last_peered":"2024-09-15T21:00:08.265411+0000","last_clean":"2024-09-15T21:00:08.265411+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:08.265411+0000","last_undegraded":"2024-09-15T21:00:08.265411+0000","last_fullsized":"2024-09-15T21:00:08.265411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902447+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:07.902447+0000","last_peered":"2024-09-15T21:00:07.902447+0000","last_clean":"2024-09-15T21:00:07.902447+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:07.902447+0000","last_undegraded":"2024-09-15T21:00:07.902447+0000","last_fullsized":"2024-09-15T21:00:07.902447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.906899+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:07.906899+0000","last_peered":"2024-09-15T21:00:07.906899+0000","last_clean":"2024-09-15T21:00:07.906899+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:07.906899+0000","last_undegraded":"2024-09-15T21:00:07.906899+0000","last_fullsized":"2024-09-15T21:00:07.906899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.928974+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:07.928974+0000","last_peered":"2024-09-15T21:00:07.928974+0000","last_clean":"2024-09-15T21:00:07.928974+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:07.928974+0000","last_undegraded":"2024-09-15T21:00:07.928974+0000","last_fullsized":"2024-09-15T21:00:07.928974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.262014+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:08.262014+0000","last_peered":"2024-09-15T21:00:08.262014+0000","last_clean":"2024-09-15T21:00:08.262014+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:08.262014+0000","last_undegraded":"2024-09-15T21:00:08.262014+0000","last_fullsized":"2024-09-15T21:00:08.262014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.905273+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:07.905273+0000","last_peered":"2024-09-15T21:00:07.905273+0000","last_clean":"2024-09-15T21:00:07.905273+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:07.905273+0000","last_undegraded":"2024-09-15T21:00:07.905273+0000","last_fullsized":"2024-09-15T21:00:07.905273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":134,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.361670+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:08.361670+0000","last_peered":"2024-09-15T21:00:08.361670+0000","last_clean":"2024-09-15T21:00:08.361670+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:08.361670+0000","last_undegraded":"2024-09-15T21:00:08.361670+0000","last_fullsized":"2024-09-15T21:00:08.361670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378631,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411335,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":623838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411334,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":685278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:14.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:14.425 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:14.442 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-15T21:00:13.523613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328220,"kb_used_data":3892,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133860,"statfs":{"total":1159641169920,"available":1159305072640,"internally_reserved":0,"allocated":3985408,"data_stored":2258254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":331855163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.641169"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.463621+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:08.463621+0000","last_peered":"2024-09-15T21:00:08.463621+0000","last_clean":"2024-09-15T21:00:08.463621+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:08.463621+0000","last_undegraded":"2024-09-15T21:00:08.463621+0000","last_fullsized":"2024-09-15T21:00:08.463621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902562+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:07.902562+0000","last_peered":"2024-09-15T21:00:07.902562+0000","last_clean":"2024-09-15T21:00:07.902562+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:07.902562+0000","last_undegraded":"2024-09-15T21:00:07.902562+0000","last_fullsized":"2024-09-15T21:00:07.902562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.265411+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:08.265411+0000","last_peered":"2024-09-15T21:00:08.265411+0000","last_clean":"2024-09-15T21:00:08.265411+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:08.265411+0000","last_undegraded":"2024-09-15T21:00:08.265411+0000","last_fullsized":"2024-09-15T21:00:08.265411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.902447+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:07.902447+0000","last_peered":"2024-09-15T21:00:07.902447+0000","last_clean":"2024-09-15T21:00:07.902447+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:07.902447+0000","last_undegraded":"2024-09-15T21:00:07.902447+0000","last_fullsized":"2024-09-15T21:00:07.902447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.906899+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:07.906899+0000","last_peered":"2024-09-15T21:00:07.906899+0000","last_clean":"2024-09-15T21:00:07.906899+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:07.906899+0000","last_undegraded":"2024-09-15T21:00:07.906899+0000","last_fullsized":"2024-09-15T21:00:07.906899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.928974+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:07.928974+0000","last_peered":"2024-09-15T21:00:07.928974+0000","last_clean":"2024-09-15T21:00:07.928974+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:07.928974+0000","last_undegraded":"2024-09-15T21:00:07.928974+0000","last_fullsized":"2024-09-15T21:00:07.928974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.262014+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:08.262014+0000","last_peered":"2024-09-15T21:00:08.262014+0000","last_clean":"2024-09-15T21:00:08.262014+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:08.262014+0000","last_undegraded":"2024-09-15T21:00:08.262014+0000","last_fullsized":"2024-09-15T21:00:08.262014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:07.905273+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:07.905273+0000","last_peered":"2024-09-15T21:00:07.905273+0000","last_clean":"2024-09-15T21:00:07.905273+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:07.905273+0000","last_undegraded":"2024-09-15T21:00:07.905273+0000","last_fullsized":"2024-09-15T21:00:07.905273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":134,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-15T21:00:08.361670+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:08.361670+0000","last_peered":"2024-09-15T21:00:08.361670+0000","last_clean":"2024-09-15T21:00:08.361670+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:08.361670+0000","last_undegraded":"2024-09-15T21:00:08.361670+0000","last_fullsized":"2024-09-15T21:00:08.361670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378631,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411335,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":623838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411334,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":685278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":94929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444038,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":94910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:14.443 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:00:14.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 25 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 07b66685-24ed-4325-b3d4-b0cceb7d7e6d 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-15T20:59:43.863405+0000 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-15T21:00:13.522395+0000 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 5 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-09-15T21:00:14.753 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client jewel 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 11.76 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 24 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 3.00 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/630056637,v1:172.21.15.27:6809/630056637] [v2:172.21.15.27:6810/630056637,v1:172.21.15.27:6811/630056637] exists,up d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/2797309965,v1:172.21.15.27:6801/2797309965] [v2:172.21.15.27:6802/2797309965,v1:172.21.15.27:6803/2797309965] exists,up 43960b2a-b250-45e3-b725-484c0751590d 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 19 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6824/1130579408,v1:172.21.15.27:6825/1130579408] [v2:172.21.15.27:6826/1130579408,v1:172.21.15.27:6827/1130579408] exists,up 9653f0df-bf71-4a94-b79a-498f067c8de8 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/2217694359,v1:172.21.15.27:6817/2217694359] [v2:172.21.15.27:6818/2217694359,v1:172.21.15.27:6819/2217694359] exists,up 9ac2adf5-f248-4838-9bd5-57d13dd53309 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/2520277109,v1:172.21.15.66:6801/2520277109] [v2:172.21.15.66:6802/2520277109,v1:172.21.15.66:6803/2520277109] exists,up 72c3acd6-6e64-4853-8823-947f944b96b5 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/1665320626,v1:172.21.15.66:6817/1665320626] [v2:172.21.15.66:6818/1665320626,v1:172.21.15.66:6819/1665320626] exists,up ad64f860-44e7-45a6-bc0d-e85458d80f39 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/1563776909,v1:172.21.15.66:6809/1563776909] [v2:172.21.15.66:6810/1563776909,v1:172.21.15.66:6811/1563776909] exists,up 4701a955-3643-4210-800d-7a86ba5155bc 2024-09-15T21:00:14.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/3067780245,v1:172.21.15.66:6825/3067780245] [v2:172.21.15.66:6826/3067780245,v1:172.21.15.66:6827/3067780245] exists,up abc724fc-25c2-4b2e-b4ba-16b61dbaf89d 2024-09-15T21:00:14.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/437424413,v1:172.21.15.92:6825/437424413] [v2:172.21.15.92:6826/437424413,v1:172.21.15.92:6827/437424413] exists,up b114589d-5020-4c1c-ad76-f07231e58d47 2024-09-15T21:00:14.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/591511703,v1:172.21.15.92:6801/591511703] [v2:172.21.15.92:6802/591511703,v1:172.21.15.92:6803/591511703] exists,up 6e4c938a-4505-41f5-90dc-416c6ceb868a 2024-09-15T21:00:14.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2398207120,v1:172.21.15.92:6809/2398207120] [v2:172.21.15.92:6810/2398207120,v1:172.21.15.92:6811/2398207120] exists,up fa725c67-48fe-45ac-bcfb-6cf3fddea7fe 2024-09-15T21:00:14.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/2517738961,v1:172.21.15.92:6817/2517738961] [v2:172.21.15.92:6818/2517738961,v1:172.21.15.92:6819/2517738961] exists,up c6fcefe9-5105-45bf-ac2d-ca8eb186316c 2024-09-15T21:00:14.770 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/630056637,v1:172.21.15.27:6809/630056637] [v2:172.21.15.27:6810/630056637,v1:172.21.15.27:6811/630056637] exists,up d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd', 'osd.1 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/2797309965,v1:172.21.15.27:6801/2797309965] [v2:172.21.15.27:6802/2797309965,v1:172.21.15.27:6803/2797309965] exists,up 43960b2a-b250-45e3-b725-484c0751590d', 'osd.2 up in weight 1 up_from 19 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6824/1130579408,v1:172.21.15.27:6825/1130579408] [v2:172.21.15.27:6826/1130579408,v1:172.21.15.27:6827/1130579408] exists,up 9653f0df-bf71-4a94-b79a-498f067c8de8', 'osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/2217694359,v1:172.21.15.27:6817/2217694359] [v2:172.21.15.27:6818/2217694359,v1:172.21.15.27:6819/2217694359] exists,up 9ac2adf5-f248-4838-9bd5-57d13dd53309', 'osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6800/2520277109,v1:172.21.15.66:6801/2520277109] [v2:172.21.15.66:6802/2520277109,v1:172.21.15.66:6803/2520277109] exists,up 72c3acd6-6e64-4853-8823-947f944b96b5', 'osd.5 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6816/1665320626,v1:172.21.15.66:6817/1665320626] [v2:172.21.15.66:6818/1665320626,v1:172.21.15.66:6819/1665320626] exists,up ad64f860-44e7-45a6-bc0d-e85458d80f39', 'osd.6 up in weight 1 up_from 17 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6808/1563776909,v1:172.21.15.66:6809/1563776909] [v2:172.21.15.66:6810/1563776909,v1:172.21.15.66:6811/1563776909] exists,up 4701a955-3643-4210-800d-7a86ba5155bc', 'osd.7 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6824/3067780245,v1:172.21.15.66:6825/3067780245] [v2:172.21.15.66:6826/3067780245,v1:172.21.15.66:6827/3067780245] exists,up abc724fc-25c2-4b2e-b4ba-16b61dbaf89d', 'osd.8 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/437424413,v1:172.21.15.92:6825/437424413] [v2:172.21.15.92:6826/437424413,v1:172.21.15.92:6827/437424413] exists,up b114589d-5020-4c1c-ad76-f07231e58d47', 'osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/591511703,v1:172.21.15.92:6801/591511703] [v2:172.21.15.92:6802/591511703,v1:172.21.15.92:6803/591511703] exists,up 6e4c938a-4505-41f5-90dc-416c6ceb868a', 'osd.10 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2398207120,v1:172.21.15.92:6809/2398207120] [v2:172.21.15.92:6810/2398207120,v1:172.21.15.92:6811/2398207120] exists,up fa725c67-48fe-45ac-bcfb-6cf3fddea7fe', 'osd.11 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/2517738961,v1:172.21.15.92:6817/2517738961] [v2:172.21.15.92:6818/2517738961,v1:172.21.15.92:6819/2517738961] exists,up c6fcefe9-5105-45bf-ac2d-ca8eb186316c'] 2024-09-15T21:00:14.770 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T21:00:14.879 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mon.a", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster": "ceph", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "admin_socket_mode": "", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_client_required": "cephx, none", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_cluster_required": "cephx", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_debug": "true", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_service_required": "cephx", 2024-09-15T21:00:14.880 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_supported": "cephx", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio": "true", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_aio_reap_max": "16", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_async_discard_threads": "0", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_block_size": "4096", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio": "false", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_enable_discard": "false", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_flock_retry": "3", 2024-09-15T21:00:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_inject_crash": "0", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring": "false", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring_hipri": "false", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T21:00:14.882 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bdev_type": "", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_allocator": "hybrid", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_buffered_io": "true", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T21:00:14.883 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_replay_recovery": "false", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_sync_write": "false", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_allocator": "hybrid", 2024-09-15T21:00:14.884 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_async_db_compaction": "true", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_create": "true", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_create": "false", 2024-09-15T21:00:14.885 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_path": "", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_db_size": "0", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_path": "", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_size": "107374182400", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_create": "false", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_path": "", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs": "true", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T21:00:14.886 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_autotune": "true", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size": "0", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T21:00:14.887 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_type": "2q", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_clone_cow": "true", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_mode": "none", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T21:00:14.888 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_freelist": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T21:00:14.889 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T21:00:14.890 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fail_eio": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T21:00:14.891 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T21:00:14.892 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size": "0", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T21:00:14.893 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_spdk_mem": "512", 2024-09-15T21:00:14.894 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_tracing": "false", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T21:00:14.895 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_write_v2": "false", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_write_v2_random": "false", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T21:00:14.896 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_cluster_require_version": "2", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_require_signatures": "false", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_require_version": "2", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_service_require_signatures": "false", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_service_require_version": "2", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cephx_sign_messages": "true", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "chdir": "", 2024-09-15T21:00:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "client_acl_type": "", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_asio_thread_count": "2", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_cache_mid": "0.750000", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_cache_size": "16384", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_caps_release_delay": "5", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_check_pool_perm": "true", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_force_sync_read": "false", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_getattr_caps": "false", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_inject_features": "", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_die_on_failed_remount": "false", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_dirsize_rbytes": "true", 2024-09-15T21:00:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "client_force_lazyio": "false", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_fs": "", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_inject_release_failure": "false", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_max_inline_size": "4096", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mds_namespace": "", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_metadata": "", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_gid": "-1", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_timeout": "300", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mount_uid": "-1", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_mountpoint": "/", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_notify_timeout": "10", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc": "true", 2024-09-15T21:00:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_max_objects": "1000", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_size": "209715200", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_permissions": "true", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_quota": "true", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_quota_df": "true", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_max_bytes": "0", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_max_periods": "4", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_readahead_min": "131072", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_reconnect_stale": "false", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_shutdown_timeout": "30", 2024-09-15T21:00:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "client_snapdir": ".snap", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_tick_interval": "1", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_trace": "", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_use_faked_inos": "false", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "client_use_random_mds": "false", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog": "false", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_graylog_port": "12201", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_monitors": "default=true", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog": "false", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "clog_to_syslog_level": "info", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "-", 2024-09-15T21:00:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_network": "", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_network_interface": "", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_isal": "false", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_level": "5", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "compressor_zstd_level": "1", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T21:00:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location": "", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location_hook": "", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_location_hook_timeout": "10", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "daemonize": "false", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "daos_pool": "tank", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T21:00:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_alienstore": "0/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asok": "1/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asok_assert_abort": "false", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_auth": "20/20", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bdev": "1/3", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bluefs": "1/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_bluestore": "1/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_buffer": "0/1", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_ceph_dedup": "0/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T21:00:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cephsqlite": "0/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_client": "0/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_compressor": "1/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_context": "0/1", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crush": "1/1", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_crypto": "1/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_cyanstore": "0/5", 2024-09-15T21:00:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_dpdk": "1/5", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_eventtrace": "1/5", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_filer": "0/1", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_filestore": "1/3", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_finisher": "1/1", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_fuse": "1/5", 2024-09-15T21:00:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_javaclient": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_journal": "1/3", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_journaler": "0/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_kstore": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_lockdep": "0/1", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mclock": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_balancer": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_locker": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_log": "1/5", 2024-09-15T21:00:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_migrator": "1/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_memstore": "1/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mgr": "2/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mgrc": "1/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_mon": "20/20", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_monc": "0/10", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_ms": "1/1", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_none": "0/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objclass": "0/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objectcacher": "0/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_objecter": "0/1", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_optracker": "0/5", 2024-09-15T21:00:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_osd": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_paxos": "20/20", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_perfcounter": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_prioritycache": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rados": "0/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd": "0/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rbd_replay": "0/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_refs": "0/0", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_reserver": "1/1", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_access": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T21:00:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_flight": "1/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_notification": "1/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rgw_sync": "1/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_rocksdb": "4/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_backref": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_cache": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_device": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_epm": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_journal": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_lba": "0/5", 2024-09-15T21:00:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_odata": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_omap": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_onode": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_t": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_seastore_tm": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_striper": "0/1", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_test": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_throttle": "1/1", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_timer": "0/1", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_tp": "0/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "debug_trace": "1/5", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "device_failure_prediction_mode": "none", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T21:00:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-15T21:00:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_graylog": "false", 2024-09-15T21:00:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_journald": "false", 2024-09-15T21:00:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_stderr": "true", 2024-09-15T21:00:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "err_to_syslog": "false", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event_tracing": "false", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_cert_file": "", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_http_port": "9926", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_key_file": "", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_prio_limit": "5", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_sort_metrics": "true", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "exporter_stats_period": "5", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "fake_statfs_for_testing": "0", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "fatal_signal_handlers": "true", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "filer_max_purge_ops": "10", 2024-09-15T21:00:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "filer_max_truncate_ops": "128", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_blackhole": "false", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_btrfs_snap": "true", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_caller_concurrency": "10", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_omap_check": "false", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_debug_verify_split": "false", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_dump_file": "", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T21:00:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fadvise": "true", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fail_eio": "true", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fd_cache_size": "128", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fiemap": "false", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_inject_stall": "0", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_parallel": "false", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_trailing": "false", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_journal_writeahead": "false", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_kill_at": "0", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T21:00:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T21:00:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T21:00:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T21:00:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T21:00:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_merge_threshold": "-10", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_odsync_write": "false", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T21:00:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_backend_path": "", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_op_threads": "2", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_punch_hole": "false", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T21:00:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_queue_max_ops": "50", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_seek_data_hole": "false", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_sloppy_crc": "false", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_splice": "false", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_split_multiple": "2", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_split_rand_factor": "20", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_update_to": "1000", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T21:00:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T21:00:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_xfs_extsize": "false", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "filestore_zfs_snap": "false", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fio_dir": "/tmp/fio", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "07b66685-24ed-4325-b3d4-b0cceb7d7e6d", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_allow_other": "true", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_big_writes": "true", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_debug": "false", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_default_permissions": "false", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_disable_pagecache": "false", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_max_write": "0", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_multithreaded": "true", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_require_active_mds": "true", 2024-09-15T21:00:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_set_user_groups": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_move": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_read": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_splice_write": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "gss_target_name": "ceph", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_file": "", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_inject_failure": "0", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_interval": "5", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "host": "smithi027", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T21:00:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "inject_early_sigterm": "false", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "jaeger_agent_port": "6799", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "jaeger_tracing_enable": "false", 2024-09-15T21:00:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_aio": "true", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_align_min_size": "65536", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_block_align": "true", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_block_size": "4096", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_dio": "true", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_discard": "false", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_force_aio": "false", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_ignore_corruption": "false", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_max_write_entries": "100", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_replay_from": "0", 2024-09-15T21:00:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_write_header_frequency": "0", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journal_zero_on_create": "false", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_prefetch_periods": "10", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_prezero_periods": "5", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "journaler_write_head_interval": "15", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "keyfile": "", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_backend": "rocksdb", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T21:00:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_max_bytes": "67108864", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_max_ops": "512", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_onode_map_size": "1024", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "kstore_sync_transaction": "false", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "librados_thread_count": "2", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "lockdep": "false", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "lockdep_force_backtrace": "false", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "log_coarse_timestamps": "true", 2024-09-15T21:00:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_flush_on_exit": "false", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_graylog_port": "12201", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_max_new": "1000", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_max_recent": "10000", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_stderr_prefix": "", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": "true", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_graylog": "false", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_journald": "false", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_stderr": "false", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_syslog": "false", 2024-09-15T21:00:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_action_on_write_error": "1", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_alternate_name_max": "8192", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_asio_thread_count": "2", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_export_pin": "true", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_interval": "10", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_max": "-1", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_max_until": "-1", 2024-09-15T21:00:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_merge_size": "50", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_mode": "0", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_bits": "3", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_size": "10000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T21:00:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_mid": "0.700000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T21:00:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_interval": "1", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T21:00:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_auth_pins": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_frag": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_scatterstat": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_debug_subtrees": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_default_dir_hash": "2", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_defer_session_stale": "true", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T21:00:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_max_entries": "0", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dir_prefetch": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_early_reply": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_enable_op_tracker": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_enforce_unique_name": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T21:00:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_health_dummy": "false", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T21:00:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_join_fs": "", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_journal_format": "1", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_create_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_dirfrag_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_export_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_import_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_link_at": "0", 2024-09-15T21:00:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_openc_at": "0", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_rename_at": "0", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_max_events": "-1", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_max_segments": "128", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_pause": "false", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_segment_size": "0", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T21:00:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_threshold": "128", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_completed_requests": "100000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_export_size": "20971520", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_file_recover": "32", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_files": "64", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_ops": "8192", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_metrics_update_interval": "2", 2024-09-15T21:00:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_min_caps_per_client": "100", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_numa_node": "-1", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_duration": "600", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_size": "20", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_op_log_threshold": "5", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_ping_grace": "15", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_ping_interval": "5", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T21:00:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_caps": "30000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_replay_interval": "1.000000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_root_ino_gid": "0", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_root_ino_uid": "0", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_scrub_stats_review_period": "1", 2024-09-15T21:00:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T21:00:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_shutdown_check": "0", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_skip_ino": "0", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_min_uid": "0", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_snap_rstat": "false", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_symlink_recovery": "true", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T21:00:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_thrash_exports": "0", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_thrash_fragments": "0", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_tick_interval": "5.000000", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_valgrind_exit": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_verify_backtrace": "1", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_verify_scatter": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_wipe_sessions": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool_debug": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T21:00:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_page_set": "false", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "memstore_page_size": "65536", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_bytes": "134217728", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_messages": "512", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_disabled_modules": "", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_max_pg_creating": "1024", 2024-09-15T21:00:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mds_messages": "128", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_mon_messages": "128", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_duration": "600", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_size": "20", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:00:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_op_log_threshold": "5", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_osd_messages": "8192", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_pool": "true", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_standby_modules": "true", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_stats_period": "5", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_stats_threshold": "5", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_tick_period": "2", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_allow_pool_delete": "true", 2024-09-15T21:00:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_bytes": "104857600", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_directed_command_retry": "2", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T21:00:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_client_target_rank": "-1", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_level": "debug", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T21:00:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_start": "false", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_compact_on_trim": "true", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_cpu_threads": "4", 2024-09-15T21:00:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_avail_crit": "5", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_avail_warn": "5", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_json": "false", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_extra_checks": "false", 2024-09-15T21:00:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_down_uptime_grace": "60", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_election_default_strategy": "1", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_election_timeout": "5.000000", 2024-09-15T21:00:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_enable_op_tracker": "true", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_fake_pool_delete": "false", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_force_quorum_join": "false", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_log_update_period": "5", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_max_detail": "50", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog": "true", 2024-09-15T21:00:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_host": "172.21.15.27,172.21.15.66,172.21.15.92", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_host_override": "", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_initial_members": "", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease": "5.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T21:00:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_full_interval": "50", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_max": "10000", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_log_max_summary": "50", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_log_epochs": "500", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_osd": "10000", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T21:00:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_memory_autotune": "true", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_memory_target": "2147483648", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_digest_period": "5", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T21:00:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_complaint_time": "30", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_duration": "600", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_size": "20", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_op_log_threshold": "5", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T21:00:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_cache_size": "500", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T21:00:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T21:00:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_report_timeout": "900", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T21:00:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T21:00:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_max_osds": "4", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_interval": "300", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_max_keys": "100", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_scrub_timeout": "300", 2024-09-15T21:00:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_session_timeout": "300", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_smart_report_timeout": "5", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_pool_size": "4", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_debug": "false", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T21:00:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_tick_interval": "5", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T21:00:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_older_version": "true", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T21:00:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "monmap": "", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "motr_tracing_enabled": "false", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_op_threads": "3", 2024-09-15T21:00:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_cm": "false", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_device_name": "", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T21:00:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_sl": "3", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_rdma_type": "ib", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_async_reap_threshold": "5", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_before_connect": "false", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_ipv4": "true", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_ipv6": "false", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_msgr1": "true", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_msgr2": "true", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_port_max": "7568", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_port_min": "6800", 2024-09-15T21:00:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_retry_count": "3", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_bind_retry_delay": "5", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_client": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mds": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mgr": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_mon": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_blackhole_osd": "false", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_client_mode": "crc secure", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T21:00:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_cluster_type": "", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_compress_secure": "false", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_crc_data": "true", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_crc_header": "true", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_bug": "true", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_old_message": "true", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T21:00:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hugepages": "", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_lro": "true", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T21:00:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_pmd": "", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_port_id": "0", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_dump_on_send": "false", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_delay_type": "", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_network_congestion": "0", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_inject_socket_failures": "0", 2024-09-15T21:00:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_max_accept_failures": "4", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_max_backoff": "15.000000", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compress_mode": "none", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_pq_min_cost": "65536", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_public_type": "", 2024-09-15T21:00:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_service_mode": "crc secure", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_nodelay": "true", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ms_type": "async+posix", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "no_config_file": "false", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T21:00:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inflight_ops": "1024", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter_timeout": "10.000000", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objectstore_blackhole": "false", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "openssl_engine_opts": "", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T21:00:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_max_ops": "4", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_agent_slop": "0.020000", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_asio_thread_count": "2", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_async_recovery_min_cost": "1", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_scan_max": "512", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backfill_scan_min": "64", 2024-09-15T21:00:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_peering": "false", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_beacon_report_interval": "300", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_duration": "30", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_blkin_trace_all": "false", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T21:00:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_class_update_on_start": "true", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_message_cap": "256", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_op_priority": "63", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_client_watch_timeout": "30", 2024-09-15T21:00:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_command_max_records": "256", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_compact_on_start": "false", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_update_on_start": "true", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T21:00:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_inject_dispatch_delay_probability": "0.100000", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_op_order": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T21:00:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_shutdown": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_trim_objects": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T21:00:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_default_notify_timeout": "30", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_ec_partial_reads": "true", 2024-09-15T21:00:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_enable_op_tracker": "true", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_info": "true", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown": "true", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T21:00:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_function_tracing": "false", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_grace": "20", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_interval": "6", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_stale": "600", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T21:00:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_journal_size": "5120", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_kill_backfill_at": "0", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T21:00:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_cache_size": "50", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_dedup": "true", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_message_max": "40", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_attr_name_len": "100", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_attr_size": "0", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_backfills": "1", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_markdown_count": "5", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_markdown_period": "600", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_name_len": "2048", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T21:00:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_object_size": "134217728", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_pgls": "1024", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_push_cost": "8388608", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_push_objects": "10", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_scrubs": "3", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T21:00:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_max_write_size": "90", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_profile": "balanced", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T21:00:14.987 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_base": "805306368", 2024-09-15T21:00:14.988 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target": "4294967296", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target_autotune": "false", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_min_recovery_priority": "0", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_report_interval": "5", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T21:00:14.989 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_num_cache_shards": "32", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_node": "-1", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objecter_finishers": "1", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore": "bluestore", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore_fuse": "false", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_objectstore_tracing": "false", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T21:00:14.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_duration": "600", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_size": "20", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_log_threshold": "5", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards": "0", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T21:00:14.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_op_thread_timeout": "15", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_open_classes_on_start": "true", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_os_flags": "0", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_peering_op_priority": "255", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_delete_priority": "5", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T21:00:14.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T21:00:14.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_crimson": "false", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_ec_fast_read": "true", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_flags": "0", 2024-09-15T21:00:14.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_min_size": "0", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_size": "2", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_default_type": "replicated", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T21:00:14.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_cost": "20971520", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active": "0", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_op_priority": "3", 2024-09-15T21:00:14.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_priority": "5", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_repair_during_recovery": "false", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T21:00:14.997 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_chunk_max": "15", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_cost": "52428800", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_end_hour": "0", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T21:00:14.998 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_priority": "5", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_retry_delay": "30", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-15T21:00:14.999 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_retry_trimming": "10", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_skip_data_digest": "false", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_smart_report_timeout": "5", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_priority": "5", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T21:00:15.000 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_target_transaction_size": "30", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T21:00:15.001 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_tracing": "false", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_use_stale_snap": "false", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_kill_at": "0", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_max_join_drift": "10", 2024-09-15T21:00:15.002 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_min": "500", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_min_wait": "0.050000", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_max": "500", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_service_trim_min": "250", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_stash_full_interval": "25", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_trim_max": "500", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "paxos_trim_min": "250", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "perf": "true", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T21:00:15.003 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "-", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrv": "", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_bind_addr": "-", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_network": "", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_network_interface": "", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_enabled": "false", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_mon_op_timeout": "0", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_osd_op_timeout": "0", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "rados_tracing": "false", 2024-09-15T21:00:15.004 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_atime_update_interval": "60", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache": "true", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T21:00:15.005 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_policy": "writearound", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_size": "33554432", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_compression_hint": "none", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_clone_format": "auto", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_data_pool": "", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_features": "61", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_format": "2", 2024-09-15T21:00:15.006 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_map_options": "", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_order": "22", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_pool": "rbd", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_stripe_count": "0", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_io_scheduler": "simple", 2024-09-15T21:00:15.007 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_order": "24", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_pool": "", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_journal_splay_width": "4", 2024-09-15T21:00:15.008 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T21:00:15.009 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T21:00:15.010 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T21:00:15.011 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_op_threads": "1", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_plugins": "", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T21:00:15.012 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T21:00:15.013 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_tracing": "false", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_validate_names": "true", 2024-09-15T21:00:15.014 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_validate_pool": "true", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "restapi_base_url": "", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "restapi_log_level": "", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_admin_entry": "admin", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_asio_assert_yielding": "false", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_backend_store": "rados", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_barbican_url": "", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_beast_enable_async": "true", 2024-09-15T21:00:15.015 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_enabled": "true", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T21:00:15.016 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_config_store": "rados", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_content_length_compat": "false", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_cross_domain_policy": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T21:00:15.017 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T21:00:15.018 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T21:00:15.019 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T21:00:15.020 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-15T21:00:15.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_log_window": "30", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T21:00:15.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_disable_s3select": "false", 2024-09-15T21:00:15.023 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T21:00:15.024 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dns_name": "", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dns_s3website_name": "", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_mdsearch": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_ops_log": "false", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T21:00:15.025 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_static_website": "false", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enable_usage_log": "false", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_expose_bucket": "false", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_extended_http_attrs": "", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_filter": "none", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T21:00:15.026 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_objs": "32", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_init_timeout": "300", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T21:00:15.027 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_password": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_project": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T21:00:15.028 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_token": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_admin_user": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T21:00:15.029 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_url": "", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_objs": "32", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_worker": "3", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T21:00:15.030 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ldap_uri": "", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_http_headers": "", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T21:00:15.031 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attr_size": "0", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_listing_results": "5000", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_notify_retries": "10", 2024-09-15T21:00:15.032 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T21:00:15.033 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T21:00:15.034 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_num_control_oids": "8", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_numa_node": "-1", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T21:00:15.035 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_op_tracing": "false", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_token": "", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_url": "", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_rados": "false", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T21:00:15.036 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_print_continue": "true", 2024-09-15T21:00:15.037 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T21:00:15.116 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T21:00:15.116 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T21:00:15.116 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rados_tracing": "false", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm": "", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm_id": "", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_region": "", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T21:00:15.117 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_request_uri": "", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T21:00:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_resolve_cname": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_run_sync_thread": "true", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T21:00:15.119 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_script_uri": "", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_service_provider_name": "", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_client_id": "", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_client_secret": "", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_entry": "sts", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_key": "sts", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T21:00:15.120 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_auth_url": "", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_custom_header": "", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_need_stats": "true", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_tenant_name": "", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_url": "", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T21:00:15.121 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_lease_period": "120", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T21:00:15.122 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_thread_pool_size": "512", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_comment": "", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_createby": "", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_encoding": "", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_flag": "false", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_origin": "", 2024-09-15T21:00:15.123 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_torrent_tracker": "", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_max_shards": "32", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_use_opa_authz": "false", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_counters_cache": "false", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T21:00:15.124 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_user_unique_email": "true", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_verify_ssl": "true", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T21:00:15.125 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone": "", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone_id": "", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup": "", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup_id": "", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_block_size": "4096", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T21:00:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T21:00:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_partition_filters": "false", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_perf": "false", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "run_dir": "/var/run/ceph", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_block_create": "true", 2024-09-15T21:00:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_device_size": "53687091200", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_full_integrity_check": "false", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T21:00:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_main_device_type": "SSD", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "seastore_segment_size": "67108864", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "set_keepcaps": "false", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "setgroup": "", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "setuser": "", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "setuser_match_path": "", 2024-09-15T21:00:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "thp": "false", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "threadpool_default_timeout": "60", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "throttler_perf_counter": "true", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tmp_dir": "/tmp", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uadk_compressor_enabled": "false", 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-15T21:00:15.131 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:15.134 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T21:00:15.279 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a: {} 2024-09-15T21:00:15.279 INFO:teuthology.orchestra.run.smithi027.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:00:15.293 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b: {} 2024-09-15T21:00:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:00:15.307 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c: {} 2024-09-15T21:00:15.307 INFO:teuthology.orchestra.run.smithi027.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:00:15.325 DEBUG:teuthology.orchestra.run.smithi066:> type ceph-objectstore-tool 2024-09-15T21:00:15.331 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:00:15.332 DEBUG:teuthology.orchestra.run.smithi092:> type ceph-objectstore-tool 2024-09-15T21:00:15.337 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:00:15.338 DEBUG:teuthology.orchestra.run.smithi027:> type ceph-objectstore-tool 2024-09-15T21:00:15.375 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:00:15.375 INFO:teuthology.run_tasks:Running task rados... 2024-09-15T21:00:15.589 INFO:tasks.rados:Beginning rados... 2024-09-15T21:00:15.589 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T21:00:15.590 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300, 'osd pool default ec fast read': True}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}} 2024-09-15T21:00:15.590 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-15T21:00:15.590 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-15T21:00:15.677 INFO:tasks.rados:joining rados 2024-09-15T21:00:15.678 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T21:00:15.678 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:15.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:15.678 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-15T21:00:15.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-15T21:00:15.679 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T21:00:15.679 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T21:00:15.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:15.679 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T21:00:15.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:15.680 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T21:00:15.680 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:00:15.680 INFO:tasks.rados:clients are ['client.0'] 2024-09-15T21:00:15.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set lrcprofile crush-failure-domain=osd k=4 l=3 m=2 name=lrcprofile plugin=lrc 2024-09-15T21:00:15.682 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:15.679+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:15.781 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:15.778+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:15.840 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:15.841 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:15.859 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:15.859 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:15.871 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:15.871 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:15.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-15T21:00:15.881 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:15.876+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:15.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:15.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:15.889 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:15.910 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:15.945 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:15.945 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:15.980 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:15.981 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:15.982 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:15.979+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:16.006 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.007 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.016 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:16.016 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:16.022 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:16.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-15T21:00:16.051 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:16.052 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:16.087 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:16.087 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:16.087 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:16.083+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:16.107 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:16.143 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:16.143 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:16.162 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.162 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:16.180 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:16.180 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:16.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-15T21:00:16.202 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:16.199+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:16.285 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:16.280+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:16.289 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.289 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:16.388 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:16.382+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:16.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-15T21:00:16.454 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.454 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.468 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:16.486 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:16.486+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:16.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-15T21:00:16.579 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.579 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.587 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:16.586+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:16.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:16.688 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:16.687+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:16.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-15T21:00:16.751 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.751 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:16.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:16.784+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:16.890 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:16.886+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:16.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-15T21:00:16.922 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:16.922 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:16.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:16.974 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:00:16.997 INFO:tasks.rados:starting run 0 out of 1 2024-09-15T21:00:16.997 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-15T21:00:16.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure lrcprofile 2024-09-15T21:00:16.998 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:16.995+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:17.065 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 reset_purged_snaps_last 2024-09-15T21:00:17.086 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.087 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:17.101 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:17.099+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:17.193 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:17.190+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:17.236 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 reset_purged_snaps_last 2024-09-15T21:00:17.266 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.267 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:17.325 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:17.323+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:17.395 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:17.391+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:17.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 reset_purged_snaps_last 2024-09-15T21:00:17.448 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.448 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.462 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:17.496 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:17.490+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:17.596 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:17.595+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:17.606 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 reset_purged_snaps_last 2024-09-15T21:00:17.625 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.625 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:17.697 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:17.692+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:17.773 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.773 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.788 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:17.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:17.828+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:17.899 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:17.899+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:17.950 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:17.951 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:17.965 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:18.001 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:17.999+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:18.091 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:18.091 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:18.101 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:18.096+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:18.105 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:18.202 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:18.202+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:18.206 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:18.272 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:18.272 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:18.286 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:18.331 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:18.327+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:18.363 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:18.363 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:18.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:18.395 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:18.404 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:18.399+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:18.405 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:18.405 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:18.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:18.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:18.429 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:18.430 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:18.450 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:18.451 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:18.471 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:18.471 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:18.490 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:18.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:18.507+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:18.512 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:18.512 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:18.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:18.554 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:18.554 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:18.575 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:18.592 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:18.592 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.70440016, 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40707127700000001, 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:18.593 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886172+0000", 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2493e-05 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886188+0000", 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6229999999999999e-05 2024-09-15T21:00:18.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886231+0000", 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2429999999999999e-05 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886265+0000", 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4202999999999998e-05 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.189845+0000", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30358023299999998 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.190242+0000", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039665399999999997 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 2", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.190684+0000", 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044248999999999997 2024-09-15T21:00:18.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293048+0000", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102364282 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293083+0000", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4403000000000002e-05 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293165+0000", 2024-09-15T21:00:18.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2216e-05 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293231+0000", 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5642999999999995e-05 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_repop_reply(client.4254.0:1 2.3 e22/21 ondisk, result = 0)", 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.396661048, 2024-09-15T21:00:18.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099164219999999997, 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193900+0000", 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7e-06 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193910+0000", 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3340000000000008e-06 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193939+0000", 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9376000000000001e-05 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193954+0000", 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5577e-05 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293025+0000", 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099070329999999998 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293039+0000", 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4303e-05 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293063+0000", 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3600000000000001e-05 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:00:18.603 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.298019+0000", 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.292540652, 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00045334100000000001, 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 2 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298019+0000", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298019+0000", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298021+0000", 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3369999999999998e-06 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298034+0000", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2666e-05 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298082+0000", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8439999999999997e-05 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298098+0000", 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5529999999999999e-05 2024-09-15T21:00:18.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298256+0000", 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015836299999999999 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.298472+0000", 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021600499999999999 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.299007+0000", 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.291552531000001, 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00037496699999999998, 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 3 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.608 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299007+0000", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299007+0000", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299009+0000", 2024-09-15T21:00:18.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0329999999999998e-06 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299028+0000", 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9029999999999999e-05 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299060+0000", 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1503e-05 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299075+0000", 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5685999999999999e-05 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299195+0000", 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000120143 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.299382+0000", 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000186572 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.611 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.704084195, 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20712275399999999, 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 5 2024-09-15T21:00:18.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886479+0000", 2024-09-15T21:00:18.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3129999999999999e-06 2024-09-15T21:00:18.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886493+0000", 2024-09-15T21:00:18.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4022999999999999e-05 2024-09-15T21:00:18.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886528+0000", 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5317000000000001e-05 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886546+0000", 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8046e-05 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886686+0000", 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014032199999999999 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886697+0000", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0913999999999999e-05 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990359+0000", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10366199199999999 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990570+0000", 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021076600000000001 2024-09-15T21:00:18.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 1", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990780+0000", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020950499999999999 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.992376+0000", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001596126 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093448+0000", 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10107226800000001 2024-09-15T21:00:18.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093531+0000", 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2645999999999994e-05 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093598+0000", 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7515999999999995e-05 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_repop_reply(client.4254.0:5 2.2 e23/21 ondisk, result = 0)", 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:07.093217+0000", 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.497342949, 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039499100000000003, 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093217+0000", 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093217+0000", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093219+0000", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0899999999999999e-06 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093230+0000", 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1194e-05 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093260+0000", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0006e-05 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093288+0000", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8402999999999999e-05 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093426+0000", 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013741599999999999 2024-09-15T21:00:18.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093440+0000", 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3867e-05 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093612+0000", 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000172015 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.689835352999999, 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.108502504, 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:18.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 7 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900727+0000", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.157e-06 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900743+0000", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5826000000000001e-05 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900779+0000", 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6332999999999999e-05 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900798+0000", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8516999999999999e-05 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900991+0000", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019261899999999999 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.901009+0000", 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8142999999999999e-05 2024-09-15T21:00:18.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905246+0000", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0042377079999999998 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905408+0000", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000161092 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 1", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905615+0000", 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020745599999999999 2024-09-15T21:00:18.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.907295+0000", 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016795250000000001 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009017+0000", 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10172268499999999 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:18.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009103+0000", 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5502999999999998e-05 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009227+0000", 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012394000000000001 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_repop_reply(client.4254.0:7 2.2 e24/21 ondisk, result = 0)", 2024-09-15T21:00:18.632 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.681840215999999, 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100521577, 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:18.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908721+0000", 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.683e-06 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908730+0000", 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5669999999999995e-06 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908756+0000", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6423000000000001e-05 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908797+0000", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.125e-05 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.008994+0000", 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10019655500000001 2024-09-15T21:00:18.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009008+0000", 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4430000000000001e-05 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009241+0000", 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023266899999999999 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.636 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.637 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:18.637 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:18.637 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:18.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:18.639 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:18.638+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:18.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:18.735+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:18.813 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:18.813 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:18.813 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:18.813 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:18.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:18.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:18.867+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:18.941 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:18.942+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:18.991 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'unique_pool_0' created 2024-09-15T21:00:19.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:00:19.008 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:19.008 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:19.008 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:19.008 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:19.008 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:19.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-15T21:00:19.024 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:19.065 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:19.063+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:19.143 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:19.142+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:19.150 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:19.151 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:19.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:19.151 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [] 2024-09-15T21:00:19.151 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:19.166 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:19.245 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:19.242+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:19.296 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:19.296 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:19.296 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:19.297 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:19.311 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:19.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:19.351+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:19.434 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:19.434 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:19.435 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:19.435 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:19.435 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:19.446 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:19.446+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:19.449 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:19.547 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:19.543+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 22.155958302999998, 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33373998100000002, 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.617 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:19.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457904+0000", 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457918+0000", 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4093999999999999e-05 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457944+0000", 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5986000000000002e-05 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457965+0000", 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0954e-05 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458117+0000", 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015210500000000001 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458129+0000", 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.149e-05 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539011+0000", 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.080882495999999998 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-09-15T21:00:19.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539035+0000", 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3439999999999999e-05 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788238+0000", 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.24920330900000001 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788633+0000", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039541700000000001 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.789053+0000", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00041907000000000001 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791200+0000", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021477969999999999 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791447+0000", 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024629799999999999 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791572+0000", 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012488899999999999 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791641+0000", 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9843000000000002e-05 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.995609437999999, 2024-09-15T21:00:19.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:00:19.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:00:19.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:00:19.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:00:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:00:19.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:19.632 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.382500938, 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30544877100000001, 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 23 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:19.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231360+0000", 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1000000000000001e-06 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231495+0000", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000134659 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231512+0000", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7297000000000001e-05 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231521+0000", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5959999999999993e-06 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234389+0000", 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002867152 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234510+0000", 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121796 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234715+0000", 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000204191 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333094+0000", 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.098379878000000004 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536645+0000", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203550593 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536729+0000", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3880000000000003e-05 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536808+0000", 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8628999999999995e-05 2024-09-15T21:00:19.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.382315178999999, 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.639 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:19.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.644 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.645 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.382130477, 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:00:19.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.652 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176854466000002, 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.653 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:00:19.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176529304999999, 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:00:19.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:00:19.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:00:19.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.667 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176488448000001, 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:00:19.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:00:19.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:00:19.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.672 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:00:19.674 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176439702, 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:00:19.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:00:19.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176124790999999, 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.682 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:00:19.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:00:19.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:19.688 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.176046724999999, 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:00:19.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:00:19.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:00:19.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:00:19.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.481588408, 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31281666299999999, 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.696 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 71 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132272+0000", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132278+0000", 2024-09-15T21:00:19.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7570000000000001e-06 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132291+0000", 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2703e-05 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132299+0000", 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0069999999999997e-06 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139452+0000", 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071530079999999998 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139562+0000", 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000110432 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139743+0000", 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018057899999999999 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142717+0000", 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002974661 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444884+0000", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216663399999999 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444969+0000", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4452999999999999e-05 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445088+0000", 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119449 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.702 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.481547774999999, 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31312449999999997, 2024-09-15T21:00:19.703 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 72 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132313+0000", 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132422+0000", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109033 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132439+0000", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7586000000000001e-05 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132449+0000", 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6199999999999994e-06 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140318+0000", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078687559999999993 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140430+0000", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112589 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140615+0000", 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018495600000000001 2024-09-15T21:00:19.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142725+0000", 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021100300000000002 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445273+0000", 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30254736500000001 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445366+0000", 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3249000000000001e-05 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445436+0000", 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0406000000000003e-05 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:19.709 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.481392152000002, 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31325695799999997, 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 73 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:19.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132469+0000", 2024-09-15T21:00:19.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132475+0000", 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4999999999999996e-06 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132493+0000", 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7669999999999999e-05 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132507+0000", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3966e-05 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141793+0000", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092860819999999993 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141919+0000", 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000126396 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142125+0000", 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020598799999999999 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.143405+0000", 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012801570000000001 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445573+0000", 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216763699999999 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.715 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445651+0000", 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8239999999999996e-05 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445725+0000", 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3329000000000002e-05 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.481307366999999, 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31343764099999999, 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 74 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.717 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132554+0000", 2024-09-15T21:00:19.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.327e-06 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132565+0000", 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0967e-05 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132584+0000", 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.984e-05 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232738+0000", 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10015367 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232890+0000", 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015175200000000001 2024-09-15T21:00:19.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233152+0000", 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026189099999999999 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233532+0000", 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038017699999999999 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239440+0000", 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005908122 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445839+0000", 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20639856300000001 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445917+0000", 2024-09-15T21:00:19.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8055999999999999e-05 2024-09-15T21:00:19.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445990+0000", 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3275999999999994e-05 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.380816855999999, 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31253853100000001, 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.725 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 76 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:19.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233045+0000", 2024-09-15T21:00:19.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233061+0000", 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6453e-05 2024-09-15T21:00:19.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233105+0000", 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4063000000000001e-05 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233123+0000", 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7937000000000001e-05 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239508+0000", 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063845940000000004 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239647+0000", 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013887599999999999 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239834+0000", 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018760200000000001 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.243229+0000", 2024-09-15T21:00:19.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033952729999999999 2024-09-15T21:00:19.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.734 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.734 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545355+0000", 2024-09-15T21:00:19.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30212512200000002 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545446+0000", 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1941999999999995e-05 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545581+0000", 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013493199999999999 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.735 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:104 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 16.301880479000001, 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30628519900000001, 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311980+0000", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312116+0000", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000136129 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312137+0000", 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1026000000000002e-05 2024-09-15T21:00:19.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312147+0000", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.594e-06 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316046+0000", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003899062 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316173+0000", 2024-09-15T21:00:19.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012724000000000001 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316430+0000", 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025629800000000002 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.317718+0000", 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287949 2024-09-15T21:00:19.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618117+0000", 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30039965099999999 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618203+0000", 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5415999999999997e-05 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618264+0000", 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1820000000000001e-05 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.743 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.385663459, 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30562285500000003, 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.744 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.745 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 116 2024-09-15T21:00:19.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.745 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228198+0000", 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.407e-06 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228209+0000", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0993e-05 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228225+0000", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6469000000000001e-05 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228241+0000", 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5739999999999998e-05 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.747 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.338640+0000", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11039951200000001 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.338852+0000", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000212062 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.339048+0000", 2024-09-15T21:00:19.748 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019553100000000001 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343126+0000", 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040778849999999998 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533696+0000", 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19057063299999999 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.749 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533767+0000", 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0049000000000006e-05 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533819+0000", 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2574000000000002e-05 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.38559102, 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.752 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:00:19.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.754 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.755 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:00:19.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:00:19.757 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 15.385420977000001, 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:19.758 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:19.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:19.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.761 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.762 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:00:19.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:19.764 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:19.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:19.766 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:19.759+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:19.869 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:19.863+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:19.897 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:19.897 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:19.897 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:19.897 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:19.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:19.971 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:19.971+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:00:20.000 INFO:teuthology.orchestra.run.smithi027.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T21:00:20.039 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.040 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:20.040 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:20.040 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:20.040 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:20.101 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:20.102+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:20.174 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:20.174+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:20.178 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.178 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:20.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:20.179 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2024-09-15T21:00:20.179 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:20.276 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:20.271+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:20.361 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:20.361 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:20.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:20.371+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:20.478 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:20.471+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:20.501 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:20.501 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:20.502 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:20.578 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:20.575+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:20.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:20.640 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:20.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:20.641 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [] 2024-09-15T21:00:20.641 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:20.697 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:20.698+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:20.751 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:20.767 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:20.768 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:20.780 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:20.780 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:20.780 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:20.780 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:20.781 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:20.778+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:20.784 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:20.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:20.800 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:20.801 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:20.821 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:20.841 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:20.841 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:20.862 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:20.862 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:20.882 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:20.875+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:20.884 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:20.904 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:20.925 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:20.925 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:20.947 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:20.947 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:20.964 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:20.964 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:20.970 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:20.970 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:20.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:20.982 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:20.975+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:21.036 DEBUG:teuthology.orchestra.run.smithi027:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight read -> 100 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight delete -> 50 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight snap_create -> 50 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight snap_remove -> 50 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight rollback -> 50 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight setattr -> 25 2024-09-15T21:00:21.056 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight rmattr -> 25 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight copy_from -> 50 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout:adding op weight append -> 100 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout:ceph version 19.3.0-4930-gfc3e0bff (fc3e0bff7468fd55478adbae7f75508b749124b6) squid (dev) 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout:Configuration: 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Number of operations: 400 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Number of objects: 50 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Max in flight operations: 16 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Object size (in bytes): 4000000 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Write stride min: 400000 2024-09-15T21:00:21.057 INFO:tasks.rados.rados.0.smithi027.stdout: Write stride max: 800000 2024-09-15T21:00:21.083 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:21.079+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:21.183 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:21.182+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:21.238 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.238 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:21.284 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:21.283+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:21.385 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:21.383+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:21.413 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.414 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:21.486 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:21.483+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:21.553 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.553 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:21.586 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:21.583+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:21.687 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:21.683+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:21.705 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.709 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:21.788 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:21.783+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:21.841 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:21.841 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:21.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:21.888 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:21.886+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:21.990 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:21.983+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:22.016 INFO:tasks.rados.rados.0.smithi027.stdout:1: write initial oid 1 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:1: seq_num 0 ranges {0=32768} 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:1: writing smithi02732947-1 from 0 to 32768 tid 1 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:2: write initial oid 2 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:2: seq_num 1 ranges {0=32768} 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:2: writing smithi02732947-2 from 0 to 32768 tid 1 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:3: write initial oid 3 2024-09-15T21:00:22.017 INFO:tasks.rados.rados.0.smithi027.stdout:3: seq_num 2 ranges {0=49152} 2024-09-15T21:00:22.018 INFO:tasks.rados.rados.0.smithi027.stdout:3: writing smithi02732947-3 from 0 to 49152 tid 1 2024-09-15T21:00:22.018 INFO:tasks.rados.rados.0.smithi027.stdout:4: write initial oid 4 2024-09-15T21:00:22.018 INFO:tasks.rados.rados.0.smithi027.stdout:4: seq_num 3 ranges {0=65536} 2024-09-15T21:00:22.019 INFO:tasks.rados.rados.0.smithi027.stdout:4: writing smithi02732947-4 from 0 to 65536 tid 1 2024-09-15T21:00:22.019 INFO:tasks.rados.rados.0.smithi027.stdout:5: write initial oid 5 2024-09-15T21:00:22.019 INFO:tasks.rados.rados.0.smithi027.stdout:5: seq_num 4 ranges {0=81920} 2024-09-15T21:00:22.020 INFO:tasks.rados.rados.0.smithi027.stdout:5: writing smithi02732947-5 from 0 to 81920 tid 1 2024-09-15T21:00:22.020 INFO:tasks.rados.rados.0.smithi027.stdout:6: write initial oid 6 2024-09-15T21:00:22.020 INFO:tasks.rados.rados.0.smithi027.stdout:6: seq_num 5 ranges {0=98304} 2024-09-15T21:00:22.021 INFO:tasks.rados.rados.0.smithi027.stdout:6: writing smithi02732947-6 from 0 to 98304 tid 1 2024-09-15T21:00:22.021 INFO:tasks.rados.rados.0.smithi027.stdout:7: write initial oid 7 2024-09-15T21:00:22.021 INFO:tasks.rados.rados.0.smithi027.stdout:7: seq_num 6 ranges {0=114688} 2024-09-15T21:00:22.022 INFO:tasks.rados.rados.0.smithi027.stdout:7: writing smithi02732947-7 from 0 to 114688 tid 1 2024-09-15T21:00:22.023 INFO:tasks.rados.rados.0.smithi027.stdout:8: write initial oid 8 2024-09-15T21:00:22.023 INFO:tasks.rados.rados.0.smithi027.stdout:8: seq_num 7 ranges {0=131072} 2024-09-15T21:00:22.024 INFO:tasks.rados.rados.0.smithi027.stdout:8: writing smithi02732947-8 from 0 to 131072 tid 1 2024-09-15T21:00:22.024 INFO:tasks.rados.rados.0.smithi027.stdout:9: write initial oid 9 2024-09-15T21:00:22.024 INFO:tasks.rados.rados.0.smithi027.stdout:9: seq_num 8 ranges {0=147456} 2024-09-15T21:00:22.026 INFO:tasks.rados.rados.0.smithi027.stdout:9: writing smithi02732947-9 from 0 to 147456 tid 1 2024-09-15T21:00:22.026 INFO:tasks.rados.rados.0.smithi027.stdout:10: write initial oid 10 2024-09-15T21:00:22.026 INFO:tasks.rados.rados.0.smithi027.stdout:10: seq_num 9 ranges {0=163840} 2024-09-15T21:00:22.028 INFO:tasks.rados.rados.0.smithi027.stdout:10: writing smithi02732947-10 from 0 to 163840 tid 1 2024-09-15T21:00:22.028 INFO:tasks.rados.rados.0.smithi027.stdout:11: write initial oid 11 2024-09-15T21:00:22.028 INFO:tasks.rados.rados.0.smithi027.stdout:11: seq_num 10 ranges {0=180224} 2024-09-15T21:00:22.030 INFO:tasks.rados.rados.0.smithi027.stdout:11: writing smithi02732947-11 from 0 to 180224 tid 1 2024-09-15T21:00:22.030 INFO:tasks.rados.rados.0.smithi027.stdout:12: write initial oid 12 2024-09-15T21:00:22.030 INFO:tasks.rados.rados.0.smithi027.stdout:12: seq_num 11 ranges {0=196608} 2024-09-15T21:00:22.031 INFO:tasks.rados.rados.0.smithi027.stdout:12: writing smithi02732947-12 from 0 to 196608 tid 1 2024-09-15T21:00:22.031 INFO:tasks.rados.rados.0.smithi027.stdout:13: write initial oid 13 2024-09-15T21:00:22.032 INFO:tasks.rados.rados.0.smithi027.stdout:13: seq_num 12 ranges {0=212992} 2024-09-15T21:00:22.033 INFO:tasks.rados.rados.0.smithi027.stdout:13: writing smithi02732947-13 from 0 to 212992 tid 1 2024-09-15T21:00:22.033 INFO:tasks.rados.rados.0.smithi027.stdout:14: write initial oid 14 2024-09-15T21:00:22.034 INFO:tasks.rados.rados.0.smithi027.stdout:14: seq_num 13 ranges {0=229376} 2024-09-15T21:00:22.034 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.034 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:00:22.036 INFO:tasks.rados.rados.0.smithi027.stdout:14: writing smithi02732947-14 from 0 to 229376 tid 1 2024-09-15T21:00:22.036 INFO:tasks.rados.rados.0.smithi027.stdout:15: write initial oid 15 2024-09-15T21:00:22.036 INFO:tasks.rados.rados.0.smithi027.stdout:15: seq_num 14 ranges {0=245760} 2024-09-15T21:00:22.038 INFO:tasks.rados.rados.0.smithi027.stdout:15: writing smithi02732947-15 from 0 to 245760 tid 1 2024-09-15T21:00:22.039 INFO:tasks.rados.rados.0.smithi027.stdout:16: write initial oid 16 2024-09-15T21:00:22.039 INFO:tasks.rados.rados.0.smithi027.stdout:16: seq_num 15 ranges {0=262144} 2024-09-15T21:00:22.040 INFO:tasks.rados.rados.0.smithi027.stdout:16: writing smithi02732947-16 from 0 to 262144 tid 1 2024-09-15T21:00:22.040 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:22.093 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:22.091+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:22.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.187+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:22.216 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.216 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:22.233 INFO:tasks.rados.rados.0.smithi027.stdout:8: finishing write tid 1 to smithi02732947-8 2024-09-15T21:00:22.292 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.287+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:22.329 INFO:tasks.rados.rados.0.smithi027.stdout:11: finishing write tid 1 to smithi02732947-11 2024-09-15T21:00:22.346 INFO:tasks.rados.rados.0.smithi027.stdout:15: finishing write tid 1 to smithi02732947-15 2024-09-15T21:00:22.352 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.353 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:22.409 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:22.407+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:22.440 INFO:tasks.rados.rados.0.smithi027.stdout:4: finishing write tid 1 to smithi02732947-4 2024-09-15T21:00:22.440 INFO:tasks.rados.rados.0.smithi027.stdout:4: finishing write tid 2 to smithi02732947-4 2024-09-15T21:00:22.441 INFO:tasks.rados.rados.0.smithi027.stdout:2: finishing write tid 1 to smithi02732947-2 2024-09-15T21:00:22.444 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.442+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:22.494 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:22.494+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:22.530 INFO:tasks.rados.rados.0.smithi027.stdout:11: finishing write tid 2 to smithi02732947-11 2024-09-15T21:00:22.531 INFO:tasks.rados.rados.0.smithi027.stdout:8: finishing write tid 2 to smithi02732947-8 2024-09-15T21:00:22.533 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.531+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:22.533 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.531+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:22.535 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.531+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:22.535 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.531+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:22.539 INFO:tasks.rados.rados.0.smithi027.stdout:2: finishing write tid 2 to smithi02732947-2 2024-09-15T21:00:22.540 INFO:tasks.rados.rados.0.smithi027.stdout:12: finishing write tid 1 to smithi02732947-12 2024-09-15T21:00:22.541 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.541 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.557 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:22.601 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:22.599+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:22.633 INFO:tasks.rados.rados.0.smithi027.stdout:1: finishing write tid 1 to smithi02732947-1 2024-09-15T21:00:22.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.631+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,4,5 2024-09-15T21:00:22.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.631+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,4,5 2024-09-15T21:00:22.645 INFO:tasks.rados.rados.0.smithi027.stdout:8: finishing write tid 3 to smithi02732947-8 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:8: oid 8 updating version 0 to 1 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:8: oid 8 updating version 1 to 2 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:8: done (15 left) 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:17: write initial oid 17 2024-09-15T21:00:22.646 INFO:tasks.rados.rados.0.smithi027.stdout:17: seq_num 16 ranges {0=278528} 2024-09-15T21:00:22.646 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.646+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:22.648 INFO:tasks.rados.rados.0.smithi027.stdout:17: writing smithi02732947-17 from 0 to 278528 tid 1 2024-09-15T21:00:22.648 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.696 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.691+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:22.724 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.724 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.733 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.727+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,4,5 2024-09-15T21:00:22.733 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:22.727+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,5 to read 0,1,4,5 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:11: finishing write tid 3 to smithi02732947-11 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:11: oid 11 updating version 0 to 1 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:11: oid 11 updating version 1 to 2 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 11 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:11: done (15 left) 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:18: write initial oid 18 2024-09-15T21:00:22.734 INFO:tasks.rados.rados.0.smithi027.stdout:18: seq_num 17 ranges {0=294912} 2024-09-15T21:00:22.738 INFO:tasks.rados.rados.0.smithi027.stdout:18: writing smithi02732947-18 from 0 to 294912 tid 1 2024-09-15T21:00:22.739 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.739 INFO:tasks.rados.rados.0.smithi027.stdout:1: finishing write tid 2 to smithi02732947-1 2024-09-15T21:00:22.740 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:22.747 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.746+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:22.747 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.746+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:22.747 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.746+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:00:22.748 INFO:tasks.rados.rados.0.smithi027.stdout:15: finishing write tid 2 to smithi02732947-15 2024-09-15T21:00:22.755 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:22.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:22.779 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:22.779 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:22.779 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T21:00:22.779 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-15T21:00:22.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T21:00:22.796 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:22.794+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:22.847 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.846+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:22.849 INFO:tasks.rados.rados.0.smithi027.stdout:16: finishing write tid 1 to smithi02732947-16 2024-09-15T21:00:22.849 INFO:tasks.rados.rados.0.smithi027.stdout:16: finishing write tid 2 to smithi02732947-16 2024-09-15T21:00:22.855 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.851+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:22.855 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.851+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:22.855 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.851+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:00:22.877 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:22.878 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:22.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:22.896 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:22.891+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:22.940 INFO:tasks.rados.rados.0.smithi027.stdout:9: finishing write tid 1 to smithi02732947-9 2024-09-15T21:00:22.940 INFO:tasks.rados.rados.0.smithi027.stdout:9: finishing write tid 2 to smithi02732947-9 2024-09-15T21:00:22.941 INFO:tasks.rados.rados.0.smithi027.stdout:10: finishing write tid 1 to smithi02732947-10 2024-09-15T21:00:22.941 INFO:tasks.rados.rados.0.smithi027.stdout:10: finishing write tid 2 to smithi02732947-10 2024-09-15T21:00:22.943 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.939+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:3: finishing write tid 1 to smithi02732947-3 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:3: finishing write tid 2 to smithi02732947-3 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:5: finishing write tid 1 to smithi02732947-5 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:5: finishing write tid 2 to smithi02732947-5 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:7: finishing write tid 1 to smithi02732947-7 2024-09-15T21:00:22.946 INFO:tasks.rados.rados.0.smithi027.stdout:7: finishing write tid 2 to smithi02732947-7 2024-09-15T21:00:22.947 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:22.943+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:22.949 INFO:tasks.rados.rados.0.smithi027.stdout:2: finishing write tid 3 to smithi02732947-2 2024-09-15T21:00:22.949 INFO:tasks.rados.rados.0.smithi027.stdout:2: oid 2 updating version 0 to 2 2024-09-15T21:00:22.949 INFO:tasks.rados.rados.0.smithi027.stdout:2: oid 2 version 2 is already newer than 1 2024-09-15T21:00:22.949 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T21:00:22.950 INFO:tasks.rados.rados.0.smithi027.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-15T21:00:22.950 INFO:tasks.rados.rados.0.smithi027.stdout:2: done (15 left) 2024-09-15T21:00:22.950 INFO:tasks.rados.rados.0.smithi027.stdout:19: write initial oid 19 2024-09-15T21:00:22.950 INFO:tasks.rados.rados.0.smithi027.stdout:19: seq_num 18 ranges {0=311296} 2024-09-15T21:00:22.950 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.950+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,4,5 2024-09-15T21:00:22.950 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:22.950+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,7 to read 0,1,4,5 2024-09-15T21:00:22.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:22.947+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:19: writing smithi02732947-19 from 0 to 311296 tid 1 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:4: finishing write tid 3 to smithi02732947-4 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:4: oid 4 updating version 0 to 1 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:4: oid 4 updating version 1 to 2 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:4: done (15 left) 2024-09-15T21:00:22.953 INFO:tasks.rados.rados.0.smithi027.stdout:20: write initial oid 20 2024-09-15T21:00:22.954 INFO:tasks.rados.rados.0.smithi027.stdout:20: seq_num 19 ranges {0=327680} 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:20: writing smithi02732947-20 from 0 to 327680 tid 1 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:16: finishing write tid 3 to smithi02732947-16 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:16: oid 16 updating version 0 to 1 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:16: oid 16 updating version 1 to 2 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:16: done (15 left) 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:21: write initial oid 21 2024-09-15T21:00:22.956 INFO:tasks.rados.rados.0.smithi027.stdout:21: seq_num 20 ranges {0=344064} 2024-09-15T21:00:22.959 INFO:tasks.rados.rados.0.smithi027.stdout:21: writing smithi02732947-21 from 0 to 344064 tid 1 2024-09-15T21:00:22.959 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:22.997 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:22.998+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:23.019 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:00:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:23.041 INFO:tasks.rados.rados.0.smithi027.stdout:17: finishing write tid 1 to smithi02732947-17 2024-09-15T21:00:23.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.039+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:23.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.039+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:23.044 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.039+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:23.044 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.039+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:00:23.045 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.039+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4 to read 0,1,4,5 2024-09-15T21:00:23.045 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.039+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:00:23.045 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.043+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:23.045 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.043+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:23.046 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.043+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:23.046 INFO:tasks.rados.rados.0.smithi027.stdout:14: finishing write tid 1 to smithi02732947-14 2024-09-15T21:00:23.046 INFO:tasks.rados.rados.0.smithi027.stdout:14: finishing write tid 2 to smithi02732947-14 2024-09-15T21:00:23.046 INFO:tasks.rados.rados.0.smithi027.stdout:1: finishing write tid 3 to smithi02732947-1 2024-09-15T21:00:23.046 INFO:tasks.rados.rados.0.smithi027.stdout:1: oid 1 updating version 0 to 1 2024-09-15T21:00:23.046 INFO:tasks.rados.rados.0.smithi027.stdout:1: oid 1 updating version 1 to 2 2024-09-15T21:00:23.047 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-15T21:00:23.047 INFO:tasks.rados.rados.0.smithi027.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-15T21:00:23.047 INFO:tasks.rados.rados.0.smithi027.stdout:1: done (15 left) 2024-09-15T21:00:23.047 INFO:tasks.rados.rados.0.smithi027.stdout:22: write initial oid 22 2024-09-15T21:00:23.047 INFO:tasks.rados.rados.0.smithi027.stdout:22: seq_num 21 ranges {0=360448} 2024-09-15T21:00:23.050 INFO:tasks.rados.rados.0.smithi027.stdout:22: writing smithi02732947-22 from 0 to 360448 tid 1 2024-09-15T21:00:23.050 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.050 INFO:tasks.rados.rados.0.smithi027.stdout:17: finishing write tid 2 to smithi02732947-17 2024-09-15T21:00:23.052 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:23.047+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:00:23.053 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:23.047+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:23.054 INFO:tasks.rados.rados.0.smithi027.stdout:15: finishing write tid 3 to smithi02732947-15 2024-09-15T21:00:23.054 INFO:tasks.rados.rados.0.smithi027.stdout:15: oid 15 updating version 0 to 1 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:15: oid 15 updating version 1 to 2 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:15: done (15 left) 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:23: write initial oid 23 2024-09-15T21:00:23.055 INFO:tasks.rados.rados.0.smithi027.stdout:23: seq_num 22 ranges {0=376832} 2024-09-15T21:00:23.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:23.057 INFO:tasks.rados.rados.0.smithi027.stdout:23: writing smithi02732947-23 from 0 to 376832 tid 1 2024-09-15T21:00:23.058 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.098 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:23.095+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:23.140 INFO:tasks.rados.rados.0.smithi027.stdout:12: finishing write tid 2 to smithi02732947-12 2024-09-15T21:00:23.142 INFO:tasks.rados.rados.0.smithi027.stdout:13: finishing write tid 1 to smithi02732947-13 2024-09-15T21:00:23.145 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:23.145 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:23.145 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:23.145 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:23.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.139+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:23.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.139+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:00:23.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.139+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5,7 to read 0,1,4,5 2024-09-15T21:00:23.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.139+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:00:23.147 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:23.147 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:23.147 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:23.147 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.143+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:23.151 INFO:tasks.rados.rados.0.smithi027.stdout:9: finishing write tid 3 to smithi02732947-9 2024-09-15T21:00:23.151 INFO:tasks.rados.rados.0.smithi027.stdout:9: oid 9 updating version 0 to 1 2024-09-15T21:00:23.151 INFO:tasks.rados.rados.0.smithi027.stdout:9: oid 9 updating version 1 to 2 2024-09-15T21:00:23.151 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-15T21:00:23.151 INFO:tasks.rados.rados.0.smithi027.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-15T21:00:23.152 INFO:tasks.rados.rados.0.smithi027.stdout:9: done (15 left) 2024-09-15T21:00:23.152 INFO:tasks.rados.rados.0.smithi027.stdout:24: write initial oid 24 2024-09-15T21:00:23.152 INFO:tasks.rados.rados.0.smithi027.stdout:24: seq_num 23 ranges {0=393216} 2024-09-15T21:00:23.152 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.147+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:23.154 INFO:tasks.rados.rados.0.smithi027.stdout:24: writing smithi02732947-24 from 0 to 393216 tid 1 2024-09-15T21:00:23.154 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.175 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:23.176 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:23.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:23.233 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.231+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:23.243 INFO:tasks.rados.rados.0.smithi027.stdout:13: finishing write tid 2 to smithi02732947-13 2024-09-15T21:00:23.245 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.239+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:23.245 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.239+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:23.245 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:23.246 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:23.247 INFO:tasks.rados.rados.0.smithi027.stdout:3: finishing write tid 3 to smithi02732947-3 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:3: oid 3 updating version 0 to 3 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:3: oid 3 updating version 3 to 4 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 3 v 4 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:5: finishing write tid 3 to smithi02732947-5 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:5: oid 5 updating version 0 to 6 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:5: oid 5 version 6 is already newer than 5 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:7: finishing write tid 3 to smithi02732947-7 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:7: oid 7 updating version 0 to 7 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:7: oid 7 updating version 7 to 8 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 7 v 8 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-15T21:00:23.248 INFO:tasks.rados.rados.0.smithi027.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:18: finishing write tid 1 to smithi02732947-18 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:18: finishing write tid 2 to smithi02732947-18 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:3: done (15 left) 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:5: done (14 left) 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:7: done (13 left) 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:25: write initial oid 25 2024-09-15T21:00:23.249 INFO:tasks.rados.rados.0.smithi027.stdout:25: seq_num 24 ranges {0=409600} 2024-09-15T21:00:23.249 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.243+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5,7 to read 0,1,4,5 2024-09-15T21:00:23.250 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.243+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:23.250 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.243+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:00:23.250 INFO:tasks.rados.rados.0.smithi027.stdout:25: writing smithi02732947-25 from 0 to 409600 tid 1 2024-09-15T21:00:23.250 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:23.247+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6 to read 0,1,4,5 2024-09-15T21:00:23.251 INFO:tasks.rados.rados.0.smithi027.stdout:26: write initial oid 26 2024-09-15T21:00:23.251 INFO:tasks.rados.rados.0.smithi027.stdout:26: seq_num 25 ranges {0=425984} 2024-09-15T21:00:23.251 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.247+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:23.253 INFO:tasks.rados.rados.0.smithi027.stdout:26: writing smithi02732947-26 from 0 to 425984 tid 1 2024-09-15T21:00:23.253 INFO:tasks.rados.rados.0.smithi027.stdout:27: write initial oid 27 2024-09-15T21:00:23.253 INFO:tasks.rados.rados.0.smithi027.stdout:27: seq_num 26 ranges {0=442368} 2024-09-15T21:00:23.256 INFO:tasks.rados.rados.0.smithi027.stdout:27: writing smithi02732947-27 from 0 to 442368 tid 1 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:14: finishing write tid 3 to smithi02732947-14 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:14: oid 14 updating version 0 to 9 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:14: oid 14 updating version 9 to 10 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 14 v 10 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:14: done (15 left) 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:28: write initial oid 28 2024-09-15T21:00:23.257 INFO:tasks.rados.rados.0.smithi027.stdout:28: seq_num 27 ranges {0=458752} 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:28: writing smithi02732947-28 from 0 to 458752 tid 1 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:10: finishing write tid 3 to smithi02732947-10 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:10: oid 10 updating version 0 to 3 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:10: oid 10 updating version 3 to 4 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:10: done (15 left) 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:29: write initial oid 29 2024-09-15T21:00:23.261 INFO:tasks.rados.rados.0.smithi027.stdout:29: seq_num 28 ranges {0=475136} 2024-09-15T21:00:23.263 INFO:tasks.rados.rados.0.smithi027.stdout:29: writing smithi02732947-29 from 0 to 475136 tid 1 2024-09-15T21:00:23.263 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.301 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:23.295+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:23.305 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:23.305 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:23.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:23.344 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.338+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:23.345 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.342+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:23.346 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.342+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:18: finishing write tid 3 to smithi02732947-18 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:18: oid 18 updating version 0 to 1 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:18: oid 18 updating version 1 to 2 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:18: done (15 left) 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:30: write initial oid 30 2024-09-15T21:00:23.350 INFO:tasks.rados.rados.0.smithi027.stdout:30: seq_num 29 ranges {0=491520} 2024-09-15T21:00:23.352 INFO:tasks.rados.rados.0.smithi027.stdout:30: writing smithi02732947-30 from 0 to 491520 tid 1 2024-09-15T21:00:23.352 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.347+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:00:23.353 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.353 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.347+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:23.365 INFO:tasks.rados.rados.0.smithi027.stdout:12: finishing write tid 3 to smithi02732947-12 2024-09-15T21:00:23.365 INFO:tasks.rados.rados.0.smithi027.stdout:12: oid 12 updating version 0 to 1 2024-09-15T21:00:23.365 INFO:tasks.rados.rados.0.smithi027.stdout:12: oid 12 updating version 1 to 2 2024-09-15T21:00:23.365 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T21:00:23.365 INFO:tasks.rados.rados.0.smithi027.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-15T21:00:23.365 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.362+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:00:23.366 INFO:tasks.rados.rados.0.smithi027.stdout:12: done (15 left) 2024-09-15T21:00:23.366 INFO:tasks.rados.rados.0.smithi027.stdout:31: write initial oid 31 2024-09-15T21:00:23.366 INFO:tasks.rados.rados.0.smithi027.stdout:31: seq_num 30 ranges {0=507904} 2024-09-15T21:00:23.369 INFO:tasks.rados.rados.0.smithi027.stdout:31: writing smithi02732947-31 from 0 to 507904 tid 1 2024-09-15T21:00:23.369 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.402 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:23.401+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:23.445 INFO:tasks.rados.rados.0.smithi027.stdout:6: finishing write tid 1 to smithi02732947-6 2024-09-15T21:00:23.445 INFO:tasks.rados.rados.0.smithi027.stdout:19: finishing write tid 1 to smithi02732947-19 2024-09-15T21:00:23.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6,7 to read 0,1,4,5 2024-09-15T21:00:23.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,4,5 2024-09-15T21:00:23.456 INFO:tasks.rados.rados.0.smithi027.stdout:17: finishing write tid 3 to smithi02732947-17 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:17: oid 17 updating version 0 to 4 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:17: oid 17 version 4 is already newer than 3 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:17: done (15 left) 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:32: write initial oid 32 2024-09-15T21:00:23.457 INFO:tasks.rados.rados.0.smithi027.stdout:32: seq_num 31 ranges {0=524288} 2024-09-15T21:00:23.466 INFO:tasks.rados.rados.0.smithi027.stdout:32: writing smithi02732947-32 from 0 to 524288 tid 1 2024-09-15T21:00:23.467 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:23.467 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:23.467 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.467 INFO:tasks.rados.rados.0.smithi027.stdout:22: finishing write tid 1 to smithi02732947-22 2024-09-15T21:00:23.467 INFO:tasks.rados.rados.0.smithi027.stdout:22: finishing write tid 2 to smithi02732947-22 2024-09-15T21:00:23.470 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.469+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:23.472 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.466+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:23.484 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:23.531 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.527+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:23.552 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:23.546+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:23.554 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:23.556 INFO:tasks.rados.rados.0.smithi027.stdout:20: finishing write tid 1 to smithi02732947-20 2024-09-15T21:00:23.556 INFO:tasks.rados.rados.0.smithi027.stdout:20: finishing write tid 2 to smithi02732947-20 2024-09-15T21:00:23.556 INFO:tasks.rados.rados.0.smithi027.stdout:21: finishing write tid 1 to smithi02732947-21 2024-09-15T21:00:23.556 INFO:tasks.rados.rados.0.smithi027.stdout:21: finishing write tid 2 to smithi02732947-21 2024-09-15T21:00:23.559 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.557+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:23.560 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.557+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:22: finishing write tid 3 to smithi02732947-22 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:22: oid 22 updating version 0 to 2 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:22: oid 22 version 2 is already newer than 1 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 2 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:22: done (15 left) 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:33: write initial oid 33 2024-09-15T21:00:23.561 INFO:tasks.rados.rados.0.smithi027.stdout:33: seq_num 32 ranges {0=540672} 2024-09-15T21:00:23.567 INFO:tasks.rados.rados.0.smithi027.stdout:33: writing smithi02732947-33 from 0 to 540672 tid 1 2024-09-15T21:00:23.567 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.567 INFO:tasks.rados.rados.0.smithi027.stdout:19: finishing write tid 2 to smithi02732947-19 2024-09-15T21:00:23.567 INFO:tasks.rados.rados.0.smithi027.stdout:6: finishing write tid 2 to smithi02732947-6 2024-09-15T21:00:23.585 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:23.585 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:23.603 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:23.599+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:23.610 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:23.611 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:23.621 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:23.622 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:23.637 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:23.646 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.645+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:23.652 INFO:tasks.rados.rados.0.smithi027.stdout:13: finishing write tid 3 to smithi02732947-13 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:13: oid 13 updating version 0 to 3 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:13: oid 13 updating version 3 to 4 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:13: done (15 left) 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:34: write initial oid 34 2024-09-15T21:00:23.653 INFO:tasks.rados.rados.0.smithi027.stdout:34: seq_num 33 ranges {0=557056} 2024-09-15T21:00:23.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.658+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:23.665 INFO:tasks.rados.rados.0.smithi027.stdout:34: writing smithi02732947-34 from 0 to 557056 tid 1 2024-09-15T21:00:23.665 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.665 INFO:tasks.rados.rados.0.smithi027.stdout:29: finishing write tid 1 to smithi02732947-29 2024-09-15T21:00:23.665 INFO:tasks.rados.rados.0.smithi027.stdout:29: finishing write tid 2 to smithi02732947-29 2024-09-15T21:00:23.705 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:23.701+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:23.711 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:23.752 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.745+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:00:23.752 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.749+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:00:23.752 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.747+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:23.755 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.753+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:23.757 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.757+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:23.757 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.757+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:23.758 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:23.757+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:6: finishing write tid 3 to smithi02732947-6 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:6: oid 6 updating version 0 to 3 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:6: oid 6 updating version 3 to 4 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:6: done (15 left) 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:35: write initial oid 35 2024-09-15T21:00:23.759 INFO:tasks.rados.rados.0.smithi027.stdout:35: seq_num 34 ranges {0=573440} 2024-09-15T21:00:23.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.755+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:00:23.762 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.758+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:23.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.759+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:00:23.763 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.758+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:23.765 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.762+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:23.765 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.762+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:23.766 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.762+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6 to read 0,1,4,5 2024-09-15T21:00:23.766 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.762+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,6 to read 0,1,4,5 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout:35: writing smithi02732947-35 from 0 to 573440 tid 1 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout:24: finishing write tid 1 to smithi02732947-24 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout:24: finishing write tid 2 to smithi02732947-24 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout:20: finishing write tid 3 to smithi02732947-20 2024-09-15T21:00:23.770 INFO:tasks.rados.rados.0.smithi027.stdout:20: oid 20 updating version 0 to 3 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:20: oid 20 updating version 3 to 4 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:20: done (15 left) 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:36: write initial oid 36 2024-09-15T21:00:23.771 INFO:tasks.rados.rados.0.smithi027.stdout:36: seq_num 35 ranges {0=589824} 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout:36: writing smithi02732947-36 from 0 to 589824 tid 1 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout:19: finishing write tid 3 to smithi02732947-19 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout:19: oid 19 updating version 0 to 2 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout:19: oid 19 version 2 is already newer than 1 2024-09-15T21:00:23.781 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 19 v 2 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T21:00:23.782 INFO:tasks.rados.rados.0.smithi027.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-15T21:00:23.782 INFO:tasks.rados.rados.0.smithi027.stdout:19: done (15 left) 2024-09-15T21:00:23.782 INFO:tasks.rados.rados.0.smithi027.stdout:37: write initial oid 37 2024-09-15T21:00:23.782 INFO:tasks.rados.rados.0.smithi027.stdout:37: seq_num 36 ranges {0=606208} 2024-09-15T21:00:23.791 INFO:tasks.rados.rados.0.smithi027.stdout:37: writing smithi02732947-37 from 0 to 606208 tid 1 2024-09-15T21:00:23.791 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.805 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:23.805+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:23.808 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:23.808 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:23.808 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:23.808 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:23.808 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:23.818 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:23.819 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:23.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:23.858 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:23.858 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:23.861 INFO:tasks.rados.rados.0.smithi027.stdout:29: finishing write tid 3 to smithi02732947-29 2024-09-15T21:00:23.861 INFO:tasks.rados.rados.0.smithi027.stdout:29: oid 29 updating version 0 to 4 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:29: oid 29 version 4 is already newer than 3 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 29 v 4 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:29: done (15 left) 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:38: write initial oid 38 2024-09-15T21:00:23.862 INFO:tasks.rados.rados.0.smithi027.stdout:38: seq_num 37 ranges {0=622592} 2024-09-15T21:00:23.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:23.859+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:23.866 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.862+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:23.866 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.862+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:38: writing smithi02732947-38 from 0 to 622592 tid 1 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:21: finishing write tid 3 to smithi02732947-21 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:21: oid 21 updating version 0 to 6 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:21: oid 21 version 6 is already newer than 5 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:21: done (15 left) 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:39: write initial oid 39 2024-09-15T21:00:23.868 INFO:tasks.rados.rados.0.smithi027.stdout:39: seq_num 38 ranges {0=638976} 2024-09-15T21:00:23.874 INFO:tasks.rados.rados.0.smithi027.stdout:39: writing smithi02732947-39 from 0 to 638976 tid 1 2024-09-15T21:00:23.874 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.874 INFO:tasks.rados.rados.0.smithi027.stdout:33: finishing write tid 1 to smithi02732947-33 2024-09-15T21:00:23.874 INFO:tasks.rados.rados.0.smithi027.stdout:28: finishing write tid 1 to smithi02732947-28 2024-09-15T21:00:23.874 INFO:tasks.rados.rados.0.smithi027.stdout:31: finishing write tid 1 to smithi02732947-31 2024-09-15T21:00:23.893 INFO:tasks.rados.rados.0.smithi027.stdout:32: finishing write tid 1 to smithi02732947-32 2024-09-15T21:00:23.894 INFO:tasks.rados.rados.0.smithi027.stdout:32: finishing write tid 2 to smithi02732947-32 2024-09-15T21:00:23.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:23.902+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:23.951 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:23.951 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:23.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:23.952 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [] 2024-09-15T21:00:23.952 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:23.961 INFO:tasks.rados.rados.0.smithi027.stdout:33: finishing write tid 2 to smithi02732947-33 2024-09-15T21:00:23.964 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.961+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:23.964 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.961+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:23.965 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:23.961+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:00:23.965 INFO:tasks.rados.rados.0.smithi027.stdout:24: finishing write tid 3 to smithi02732947-24 2024-09-15T21:00:23.965 INFO:tasks.rados.rados.0.smithi027.stdout:24: oid 24 updating version 0 to 6 2024-09-15T21:00:23.965 INFO:tasks.rados.rados.0.smithi027.stdout:24: oid 24 version 6 is already newer than 5 2024-09-15T21:00:23.965 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T21:00:23.966 INFO:tasks.rados.rados.0.smithi027.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-15T21:00:23.966 INFO:tasks.rados.rados.0.smithi027.stdout:24: done (15 left) 2024-09-15T21:00:23.966 INFO:tasks.rados.rados.0.smithi027.stdout:40: write initial oid 40 2024-09-15T21:00:23.966 INFO:tasks.rados.rados.0.smithi027.stdout:40: seq_num 39 ranges {0=671744} 2024-09-15T21:00:23.967 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:23.977 INFO:tasks.rados.rados.0.smithi027.stdout:40: writing smithi02732947-40 from 0 to 671744 tid 1 2024-09-15T21:00:23.977 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:23.977 INFO:tasks.rados.rados.0.smithi027.stdout:28: finishing write tid 2 to smithi02732947-28 2024-09-15T21:00:23.977 INFO:tasks.rados.rados.0.smithi027.stdout:31: finishing write tid 2 to smithi02732947-31 2024-09-15T21:00:24.003 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:24.004 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:24.007 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:24.005+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:24.023 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T21:00:24.058 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.054+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:24.058 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.054+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:00:24.059 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.054+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:00:24.061 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.058+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:24.061 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.058+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:24.061 INFO:tasks.rados.rados.0.smithi027.stdout:23: finishing write tid 1 to smithi02732947-23 2024-09-15T21:00:24.061 INFO:tasks.rados.rados.0.smithi027.stdout:23: finishing write tid 2 to smithi02732947-23 2024-09-15T21:00:24.062 INFO:tasks.rados.rados.0.smithi027.stdout:26: finishing write tid 1 to smithi02732947-26 2024-09-15T21:00:24.062 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.058+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,7 to read 0,1,4,5 2024-09-15T21:00:24.062 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.058+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,4,5 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:28: finishing write tid 3 to smithi02732947-28 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:28: oid 28 updating version 0 to 1 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:28: oid 28 updating version 1 to 2 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 28 v 2 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:28: done (15 left) 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:41: write initial oid 41 2024-09-15T21:00:24.063 INFO:tasks.rados.rados.0.smithi027.stdout:41: seq_num 40 ranges {0=688128} 2024-09-15T21:00:24.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.059+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:24.068 INFO:tasks.rados.rados.0.smithi027.stdout:41: writing smithi02732947-41 from 0 to 688128 tid 1 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:33: finishing write tid 3 to smithi02732947-33 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:33: oid 33 updating version 0 to 4 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:33: oid 33 version 4 is already newer than 3 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 33 v 4 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:33: done (15 left) 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:42: write initial oid 42 2024-09-15T21:00:24.069 INFO:tasks.rados.rados.0.smithi027.stdout:42: seq_num 41 ranges {0=704512} 2024-09-15T21:00:24.074 INFO:tasks.rados.rados.0.smithi027.stdout:42: writing smithi02732947-42 from 0 to 704512 tid 1 2024-09-15T21:00:24.074 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.087 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:24.088 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:24.108 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:24.103+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:24.109 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:24.109 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:24.137 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:24.137 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:24.141 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 32/28 ECSubWriteReply(tid=20, last_complete=32'6, committed=1, applied=1))", 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.080138021000000004, 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.080157494999999995, 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060833+0000", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060837+0000", 2024-09-15T21:00:24.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2919999999999997e-06 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060852+0000", 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:24.144 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:00:24.145 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:24.157 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:24.162 INFO:tasks.rados.rados.0.smithi027.stdout:25: finishing write tid 1 to smithi02732947-25 2024-09-15T21:00:24.162 INFO:tasks.rados.rados.0.smithi027.stdout:25: finishing write tid 2 to smithi02732947-25 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:23: finishing write tid 3 to smithi02732947-23 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:23: oid 23 updating version 0 to 5 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:23: oid 23 updating version 5 to 6 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 23 v 6 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:26: finishing write tid 2 to smithi02732947-26 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:23: done (15 left) 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:43: write initial oid 43 2024-09-15T21:00:24.164 INFO:tasks.rados.rados.0.smithi027.stdout:43: seq_num 42 ranges {0=720896} 2024-09-15T21:00:24.166 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:24.166 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:24.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.159+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:24.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.163+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:00:24.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.163+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:24.176 INFO:tasks.rados.rados.0.smithi027.stdout:43: writing smithi02732947-43 from 0 to 720896 tid 1 2024-09-15T21:00:24.176 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.176 INFO:tasks.rados.rados.0.smithi027.stdout:30: finishing write tid 1 to smithi02732947-30 2024-09-15T21:00:24.176 INFO:tasks.rados.rados.0.smithi027.stdout:30: finishing write tid 2 to smithi02732947-30 2024-09-15T21:00:24.176 INFO:tasks.rados.rados.0.smithi027.stdout:27: finishing write tid 1 to smithi02732947-27 2024-09-15T21:00:24.177 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.170+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:24.177 INFO:tasks.rados.rados.0.smithi027.stdout:34: finishing write tid 1 to smithi02732947-34 2024-09-15T21:00:24.177 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.174+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:24.178 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.174+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:00:24.179 INFO:tasks.rados.rados.0.smithi027.stdout:34: finishing write tid 2 to smithi02732947-34 2024-09-15T21:00:24.182 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.178+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,5 to read 0,1,4,5 2024-09-15T21:00:24.183 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:24.179+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:24.184 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:24.179+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:24.196 INFO:tasks.rados.rados.0.smithi027.stdout:32: finishing write tid 3 to smithi02732947-32 2024-09-15T21:00:24.196 INFO:tasks.rados.rados.0.smithi027.stdout:32: oid 32 updating version 0 to 1 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:32: oid 32 updating version 1 to 2 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 32 v 2 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:32: done (15 left) 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:44: write initial oid 44 2024-09-15T21:00:24.197 INFO:tasks.rados.rados.0.smithi027.stdout:44: seq_num 43 ranges {0=737280} 2024-09-15T21:00:24.202 INFO:tasks.rados.rados.0.smithi027.stdout:44: writing smithi02732947-44 from 0 to 737280 tid 1 2024-09-15T21:00:24.202 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.208 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.203+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:24.267 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.262+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:24.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.263+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:24.277 INFO:tasks.rados.rados.0.smithi027.stdout:31: finishing write tid 3 to smithi02732947-31 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:31: oid 31 updating version 0 to 3 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:31: oid 31 updating version 3 to 4 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:31: done (15 left) 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:45: write initial oid 45 2024-09-15T21:00:24.278 INFO:tasks.rados.rados.0.smithi027.stdout:45: seq_num 44 ranges {0=753664} 2024-09-15T21:00:24.290 INFO:tasks.rados.rados.0.smithi027.stdout:45: writing smithi02732947-45 from 0 to 753664 tid 1 2024-09-15T21:00:24.291 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:24.299 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:24.299 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:24.299 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:24.299 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:24.310 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:24.303+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:24.314 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:24.368 INFO:tasks.rados.rados.0.smithi027.stdout:27: finishing write tid 2 to smithi02732947-27 2024-09-15T21:00:24.369 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.363+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:00:24.370 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.363+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6,7 to read 0,1,4,5 2024-09-15T21:00:24.370 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.363+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,4,5 2024-09-15T21:00:24.381 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:24.375+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4 to read 0,1,4,5 2024-09-15T21:00:24.383 INFO:tasks.rados.rados.0.smithi027.stdout:39: finishing write tid 1 to smithi02732947-39 2024-09-15T21:00:24.410 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.406+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:24.468 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.462+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:00:24.468 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.462+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:00:24.469 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.466+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:00:24.470 INFO:tasks.rados.rados.0.smithi027.stdout:30: finishing write tid 3 to smithi02732947-30 2024-09-15T21:00:24.470 INFO:tasks.rados.rados.0.smithi027.stdout:30: oid 30 updating version 0 to 9 2024-09-15T21:00:24.470 INFO:tasks.rados.rados.0.smithi027.stdout:30: oid 30 updating version 9 to 10 2024-09-15T21:00:24.470 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 10 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T21:00:24.470 INFO:tasks.rados.rados.0.smithi027.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-15T21:00:24.470 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.466+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:24.471 INFO:tasks.rados.rados.0.smithi027.stdout:30: done (15 left) 2024-09-15T21:00:24.471 INFO:tasks.rados.rados.0.smithi027.stdout:46: write initial oid 46 2024-09-15T21:00:24.471 INFO:tasks.rados.rados.0.smithi027.stdout:46: seq_num 45 ranges {0=770048} 2024-09-15T21:00:24.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.467+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:24.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.467+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:24.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:24.467+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout:46: writing smithi02732947-46 from 0 to 770048 tid 1 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout:25: finishing write tid 3 to smithi02732947-25 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout:25: oid 25 updating version 0 to 5 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout:25: oid 25 updating version 5 to 6 2024-09-15T21:00:24.483 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:27: finishing write tid 3 to smithi02732947-27 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:27: oid 27 updating version 0 to 8 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:27: oid 27 version 8 is already newer than 7 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 27 v 8 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:25: done (15 left) 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:27: done (14 left) 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:47: write initial oid 47 2024-09-15T21:00:24.484 INFO:tasks.rados.rados.0.smithi027.stdout:47: seq_num 46 ranges {0=786432} 2024-09-15T21:00:24.498 INFO:tasks.rados.rados.0.smithi027.stdout:47: writing smithi02732947-47 from 0 to 786432 tid 1 2024-09-15T21:00:24.498 INFO:tasks.rados.rados.0.smithi027.stdout:48: write initial oid 48 2024-09-15T21:00:24.498 INFO:tasks.rados.rados.0.smithi027.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.614370260000001, 2024-09-15T21:00:24.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40707127700000001, 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:24.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886159+0000", 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886172+0000", 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2493e-05 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886188+0000", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6229999999999999e-05 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886231+0000", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2429999999999999e-05 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:05.886265+0000", 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4202999999999998e-05 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.189845+0000", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30358023299999998 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.190242+0000", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039665399999999997 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 2", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.190684+0000", 2024-09-15T21:00:24.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00044248999999999997 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293048+0000", 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102364282 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293083+0000", 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4403000000000002e-05 2024-09-15T21:00:24.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293165+0000", 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2216e-05 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293231+0000", 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5642999999999995e-05 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_repop_reply(client.4254.0:1 2.3 e22/21 ondisk, result = 0)", 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 18.306631148000001, 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099164219999999997, 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193898+0000", 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193900+0000", 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7e-06 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193910+0000", 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3340000000000008e-06 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193939+0000", 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9376000000000001e-05 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.193954+0000", 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5577e-05 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293025+0000", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099070329999999998 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293039+0000", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4303e-05 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.293063+0000", 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3600000000000001e-05 2024-09-15T21:00:24.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 17.614054294999999, 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20712275399999999, 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:24.515 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 5 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886475+0000", 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886479+0000", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3129999999999999e-06 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886493+0000", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4022999999999999e-05 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886528+0000", 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5317000000000001e-05 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886546+0000", 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8046e-05 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886686+0000", 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014032199999999999 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:24.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.886697+0000", 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0913999999999999e-05 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990359+0000", 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10366199199999999 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990570+0000", 2024-09-15T21:00:24.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021076600000000001 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 1", 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.990780+0000", 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020950499999999999 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:06.992376+0000", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001596126 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093448+0000", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10107226800000001 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093531+0000", 2024-09-15T21:00:24.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2645999999999994e-05 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.093598+0000", 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7515999999999995e-05 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4254.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:24.522 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 16.599805452999998, 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.108502504, 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4254", 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/416303025", 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 7 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900724+0000", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900727+0000", 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.157e-06 2024-09-15T21:00:24.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900743+0000", 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5826000000000001e-05 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900779+0000", 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6332999999999999e-05 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900798+0000", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8516999999999999e-05 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.900991+0000", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019261899999999999 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.901009+0000", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8142999999999999e-05 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905246+0000", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0042377079999999998 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905408+0000", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000161092 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 1", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.905615+0000", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020745599999999999 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:24.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.907295+0000", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016795250000000001 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009017+0000", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10172268499999999 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009103+0000", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5502999999999998e-05 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009227+0000", 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012394000000000001 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_repop_reply(client.4254.0:7 2.2 e24/21 ondisk, result = 0)", 2024-09-15T21:00:24.529 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 16.591810316, 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100521577, 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908719+0000", 2024-09-15T21:00:24.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908721+0000", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.683e-06 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908730+0000", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5669999999999995e-06 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908756+0000", 2024-09-15T21:00:24.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6423000000000001e-05 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:07.908797+0000", 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.125e-05 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.008994+0000", 2024-09-15T21:00:24.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10019655500000001 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009008+0000", 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4430000000000001e-05 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:08.009241+0000", 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023266899999999999 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 32/28 ECSubWriteReply(tid=23, last_complete=32'6, committed=1, applied=1))", 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.860400+0000", 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.64012982200000001, 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10442733999999999, 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860400+0000", 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860400+0000", 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860402+0000", 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0660000000000002e-06 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860410+0000", 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5080000000000005e-06 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860440+0000", 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9450000000000001e-05 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860453+0000", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3111e-05 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.964788+0000", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10433500499999999 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.964827+0000", 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9199999999999997e-05 2024-09-15T21:00:24.537 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.cs0 32/28 ECSubReadReply(tid=24, attrs_read=0))", 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.861176+0000", 2024-09-15T21:00:24.538 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.63935373399999995, 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103762298, 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861176+0000", 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861176+0000", 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861177+0000", 2024-09-15T21:00:24.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5000000000000001e-07 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861210+0000", 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3541000000000003e-05 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861250+0000", 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9341999999999997e-05 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.861259+0000", 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5510000000000005e-06 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.964885+0000", 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103626181 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.964938+0000", 2024-09-15T21:00:24.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2732999999999998e-05 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.cs0 32/28 ECSubReadReply(tid=24, attrs_read=0))", 2024-09-15T21:00:24.546 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.862220+0000", 2024-09-15T21:00:24.547 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.63830943600000001, 2024-09-15T21:00:24.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102837146, 2024-09-15T21:00:24.547 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.547 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862220+0000", 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862220+0000", 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862221+0000", 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2600000000000002e-07 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862234+0000", 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2690999999999999e-05 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862251+0000", 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7187e-05 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862259+0000", 2024-09-15T21:00:24.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3119999999999994e-06 2024-09-15T21:00:24.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965009+0000", 2024-09-15T21:00:24.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10274943 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965057+0000", 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.88e-05 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.554 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 32/28 ECSubWriteReply(tid=23, last_complete=32'6, committed=1, applied=1))", 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.862331+0000", 2024-09-15T21:00:24.555 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.63819829299999997, 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202430569, 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862331+0000", 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862331+0000", 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862332+0000", 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5999999999999991e-07 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862338+0000", 2024-09-15T21:00:24.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6540000000000004e-06 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.862354+0000", 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5860999999999999e-05 2024-09-15T21:00:24.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962443+0000", 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100089179 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064728+0000", 2024-09-15T21:00:24.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102285345 2024-09-15T21:00:24.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064762+0000", 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3569999999999999e-05 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/28 ECSubWrite(tid=6, reqid=client.4641.0:98, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.962169+0000", 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.53836089399999998, 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101291452, 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.564 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962169+0000", 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962169+0000", 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962171+0000", 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3429999999999998e-06 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962200+0000", 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9067000000000001e-05 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962249+0000", 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9348999999999998e-05 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962276+0000", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6780000000000001e-05 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.062479+0000", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100203151 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.062487+0000", 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9200000000000004e-06 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.063407+0000", 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00091997900000000002 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.063460+0000", 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2862999999999999e-05 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.9s4 32/28 ECSubRead(tid=7, to_read={3:9109fcd4:::smithi02732947-33:head=0,4096,0}, subchunks={3:9109fcd4:::smithi02732947-33:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.962332+0000", 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.53819775199999997, 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100870481, 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962332+0000", 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962332+0000", 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962333+0000", 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0330000000000001e-06 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962348+0000", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5087e-05 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962381+0000", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3463000000000001e-05 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962407+0000", 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5327e-05 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.062791+0000", 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100384634 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.063202+0000", 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000410937 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 32/28 ECSubWriteReply(tid=22, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.962462+0000", 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.53806810299999996, 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.098402555000000003, 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962462+0000", 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962462+0000", 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962463+0000", 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7399999999999991e-07 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962470+0000", 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9500000000000001e-06 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962491+0000", 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0755999999999999e-05 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962506+0000", 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4589999999999999e-05 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060845+0000", 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.098339420999999996 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060864+0000", 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8864000000000001e-05 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.cs0 32/28 ECSubReadReply(tid=24, attrs_read=0))", 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.962550+0000", 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.53797954299999995, 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102312731, 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962550+0000", 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962550+0000", 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962551+0000", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.33e-06 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962574+0000", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.266e-05 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962598+0000", 2024-09-15T21:00:24.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4206000000000002e-05 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962614+0000", 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5452999999999999e-05 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064809+0000", 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102195466 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064863+0000", 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3616000000000003e-05 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 32/28 ECSubWriteReply(tid=23, last_complete=32'6, committed=1, applied=1))", 2024-09-15T21:00:24.584 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.962641+0000", 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.53788830099999996, 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10269125599999999, 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962641+0000", 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962641+0000", 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962642+0000", 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6700000000000003e-07 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962650+0000", 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6559999999999992e-06 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962669+0000", 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8749999999999998e-05 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962683+0000", 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4507e-05 2024-09-15T21:00:24.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065204+0000", 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10252109099999999 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065333+0000", 2024-09-15T21:00:24.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012848499999999999 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s4 32/28 ECSubWrite(tid=27, reqid=client.4641.0:115, at_version=32'11, trim_to=0'0, roll_forward_to=30'10))", 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.977017+0000", 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.52351270699999997, 2024-09-15T21:00:24.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10302702699999999, 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977017+0000", 2024-09-15T21:00:24.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977017+0000", 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977019+0000", 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1239999999999999e-06 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977449+0000", 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042992200000000001 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977462+0000", 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3149000000000001e-05 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977468+0000", 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3350000000000003e-06 2024-09-15T21:00:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077687+0000", 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100218486 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077717+0000", 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9632999999999999e-05 2024-09-15T21:00:24.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.079960+0000", 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022434909999999998 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.080044+0000", 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3887000000000005e-05 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s5 32/28 ECSubWrite(tid=4, reqid=client.4641.0:103, at_version=32'3, trim_to=0'0, roll_forward_to=32'2))", 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.994289+0000", 2024-09-15T21:00:24.597 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.50624064899999999, 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102110119, 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994289+0000", 2024-09-15T21:00:24.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994289+0000", 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994290+0000", 2024-09-15T21:00:24.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8800000000000002e-07 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994370+0000", 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0839000000000003e-05 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994387+0000", 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6813999999999999e-05 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.994401+0000", 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3339000000000001e-05 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094604+0000", 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100203638 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094620+0000", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5435999999999999e-05 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.096324+0000", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017048199999999999 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.096399+0000", 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4645000000000001e-05 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.602 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 32/28 ECSubWriteReply(tid=20, last_complete=32'6, committed=1, applied=1))", 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.439697644, 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10045402, 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060832+0000", 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060833+0000", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060837+0000", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2919999999999997e-06 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.060852+0000", 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.160980+0000", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012810599999999 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161117+0000", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013707300000000001 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161286+0000", 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016875899999999999 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.606 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s0 32/28 ECSubRead(tid=25, to_read={3:a5da2594:::smithi02732947-25:head=0,4096,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.161603+0000", 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.33892674099999998, 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101261767, 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161603+0000", 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161603+0000", 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161603+0000", 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161603+0000", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161619+0000", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6110999999999999e-05 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161631+0000", 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2014000000000001e-05 2024-09-15T21:00:24.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262457+0000", 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100825818 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262865+0000", 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040782399999999999 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.610 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 33/28 ECSubReadReply(tid=25, attrs_read=0))", 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162913+0000", 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.337616794, 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104608865, 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162913+0000", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162913+0000", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162914+0000", 2024-09-15T21:00:24.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162926+0000", 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2605e-05 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162940+0000", 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3738e-05 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162949+0000", 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7900000000000005e-06 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267297+0000", 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10434823 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267522+0000", 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022481199999999999 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 33/28 ECSubWrite(tid=14, reqid=client.4641.0:89, at_version=0'0, trim_to=0'0, roll_forward_to=32'10))", 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163509+0000", 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.337020563, 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101756203, 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163509+0000", 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163509+0000", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163509+0000", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163525+0000", 2024-09-15T21:00:24.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5815999999999999e-05 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163543+0000", 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7334e-05 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163553+0000", 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0295999999999999e-05 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:24.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263770+0000", 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100217313 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263798+0000", 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7673000000000001e-05 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265201+0000", 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001402913 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:24.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265265+0000", 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4504000000000004e-05 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:24.619 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:24.621 INFO:tasks.rados.rados.0.smithi027.stdout:48: writing smithi02732947-48 from 0 to 425984 tid 1 2024-09-15T21:00:24.621 INFO:tasks.rados.rados.0.smithi027.stdout:48: writing smithi02732947-48 from 425984 to 802816 tid 2 2024-09-15T21:00:24.621 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.622 INFO:tasks.rados.rados.0.smithi027.stdout:39: finishing write tid 2 to smithi02732947-39 2024-09-15T21:00:24.623 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.566+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:00:24.623 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.566+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,4,5 2024-09-15T21:00:24.623 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:24.623 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:24.575+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,5 to read 0,1,4,5 2024-09-15T21:00:24.624 INFO:tasks.rados.rados.0.smithi027.stdout:35: finishing write tid 1 to smithi02732947-35 2024-09-15T21:00:24.624 INFO:tasks.rados.rados.0.smithi027.stdout:36: finishing write tid 1 to smithi02732947-36 2024-09-15T21:00:24.624 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.582+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:24.624 INFO:tasks.rados.rados.0.smithi027.stdout:43: finishing write tid 1 to smithi02732947-43 2024-09-15T21:00:24.625 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:24.617+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:24.667 INFO:tasks.rados.rados.0.smithi027.stdout:40: finishing write tid 1 to smithi02732947-40 2024-09-15T21:00:24.667 INFO:tasks.rados.rados.0.smithi027.stdout:40: finishing write tid 2 to smithi02732947-40 2024-09-15T21:00:24.667 INFO:tasks.rados.rados.0.smithi027.stdout:35: finishing write tid 2 to smithi02732947-35 2024-09-15T21:00:24.673 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.666+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:26: finishing write tid 3 to smithi02732947-26 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:26: oid 26 updating version 0 to 8 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:26: oid 26 version 8 is already newer than 7 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 26 v 8 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:26: done (15 left) 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:49: write initial oid 49 2024-09-15T21:00:24.674 INFO:tasks.rados.rados.0.smithi027.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-15T21:00:24.681 INFO:tasks.rados.rados.0.smithi027.stdout:49: writing smithi02732947-49 from 0 to 442368 tid 1 2024-09-15T21:00:24.687 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.682+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout:49: writing smithi02732947-49 from 442368 to 819200 tid 2 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout:37: finishing write tid 1 to smithi02732947-37 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout:34: finishing write tid 3 to smithi02732947-34 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout:34: oid 34 updating version 0 to 2 2024-09-15T21:00:24.689 INFO:tasks.rados.rados.0.smithi027.stdout:34: oid 34 version 2 is already newer than 1 2024-09-15T21:00:24.690 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 34 v 2 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T21:00:24.690 INFO:tasks.rados.rados.0.smithi027.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-15T21:00:24.690 INFO:tasks.rados.rados.0.smithi027.stdout:34: done (15 left) 2024-09-15T21:00:24.690 INFO:tasks.rados.rados.0.smithi027.stdout:50: write initial oid 50 2024-09-15T21:00:24.690 INFO:tasks.rados.rados.0.smithi027.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-15T21:00:24.691 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.686+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:24.692 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.686+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:24.692 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.686+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:00:24.696 INFO:tasks.rados.rados.0.smithi027.stdout:50: writing smithi02732947-50 from 0 to 458752 tid 1 2024-09-15T21:00:24.703 INFO:tasks.rados.rados.0.smithi027.stdout:50: writing smithi02732947-50 from 458752 to 835584 tid 2 2024-09-15T21:00:24.703 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.703 INFO:tasks.rados.rados.0.smithi027.stdout:45: finishing write tid 1 to smithi02732947-45 2024-09-15T21:00:24.704 INFO:tasks.rados.rados.0.smithi027.stdout:45: finishing write tid 2 to smithi02732947-45 2024-09-15T21:00:24.721 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.718+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:24.767 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.762+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:24.767 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.762+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:24.767 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.762+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:24.767 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.762+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,6 to read 0,1,4,5 2024-09-15T21:00:24.779 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.775+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:24.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.775+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:43: finishing write tid 2 to smithi02732947-43 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:40: finishing write tid 3 to smithi02732947-40 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:40: oid 40 updating version 0 to 5 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:40: oid 40 updating version 5 to 6 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-15T21:00:24.789 INFO:tasks.rados.rados.0.smithi027.stdout:40: done (15 left) 2024-09-15T21:00:24.790 INFO:tasks.rados.rados.0.smithi027.stdout:51: copy_from oid 2 from oid 11 current snap is 0 2024-09-15T21:00:24.790 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.804 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:24.804 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:24.804 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:24.804 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:24.819 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:24.822 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:24.818+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:24.880 INFO:tasks.rados.rados.0.smithi027.stdout:37: finishing write tid 2 to smithi02732947-37 2024-09-15T21:00:24.881 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.875+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6 to read 0,1,4,5 2024-09-15T21:00:24.881 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.875+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:00:24.882 INFO:tasks.rados.rados.0.smithi027.stdout:35: finishing write tid 3 to smithi02732947-35 2024-09-15T21:00:24.882 INFO:tasks.rados.rados.0.smithi027.stdout:35: oid 35 updating version 0 to 4 2024-09-15T21:00:24.882 INFO:tasks.rados.rados.0.smithi027.stdout:35: oid 35 version 4 is already newer than 3 2024-09-15T21:00:24.882 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 35 v 4 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-15T21:00:24.882 INFO:tasks.rados.rados.0.smithi027.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-15T21:00:24.883 INFO:tasks.rados.rados.0.smithi027.stdout:35: done (15 left) 2024-09-15T21:00:24.883 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 25 current snap is 0 2024-09-15T21:00:24.883 INFO:tasks.rados.rados.0.smithi027.stdout:52: seq_num 50 ranges {409600=475136,884736=376832} 2024-09-15T21:00:24.883 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.879+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:24.883 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.879+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:00:24.883 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:24.879+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:00:24.887 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.882+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:24.897 INFO:tasks.rados.rados.0.smithi027.stdout:52: writing smithi02732947-25 from 409600 to 884736 tid 1 2024-09-15T21:00:24.903 INFO:tasks.rados.rados.0.smithi027.stdout:52: writing smithi02732947-25 from 884736 to 1261568 tid 2 2024-09-15T21:00:24.903 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:24.923 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:24.918+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:24.944 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:24.945 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:24.945 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:24.945 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:24.945 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:24.959 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:24.983 INFO:tasks.rados.rados.0.smithi027.stdout:37: finishing write tid 3 to smithi02732947-37 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:37: oid 37 updating version 0 to 6 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:37: oid 37 version 6 is already newer than 5 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:37: done (15 left) 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:53: rmattr oid 30 current snap is 0 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:53: done (15 left) 2024-09-15T21:00:24.984 INFO:tasks.rados.rados.0.smithi027.stdout:54: delete oid 3 current snap is 0 2024-09-15T21:00:25.024 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.021+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:25.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:00:25.083 INFO:tasks.rados.rados.0.smithi027.stdout:41: finishing write tid 1 to smithi02732947-41 2024-09-15T21:00:25.083 INFO:tasks.rados.rados.0.smithi027.stdout:41: finishing write tid 2 to smithi02732947-41 2024-09-15T21:00:25.091 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:25.091 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 32/28 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.1226669069999999, 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.130100361, 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965252+0000", 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0419999999999999e-06 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965258+0000", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7620000000000001e-06 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965274+0000", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6050999999999999e-05 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065378+0000", 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10010451400000001 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094985+0000", 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029606517999999998 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095351+0000", 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036647399999999999 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.095 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 32/28 ECSubWrite(tid=27, reqid=client.4641.0:115, at_version=32'11, trim_to=0'0, roll_forward_to=30'10))", 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.1109673440000001, 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.117700763, 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976952+0000", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.035e-06 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977149+0000", 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000197061 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977164+0000", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5732000000000001e-05 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977175+0000", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0995999999999999e-05 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077321+0000", 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10014511399999999 2024-09-15T21:00:25.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077329+0000", 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9630000000000004e-06 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094485+0000", 2024-09-15T21:00:25.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017155026 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094651+0000", 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016683600000000001 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 32/28 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.022478035, 2024-09-15T21:00:25.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113605911, 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065441+0000", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1030000000000001e-06 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065449+0000", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1170000000000004e-06 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065471+0000", 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1846000000000001e-05 2024-09-15T21:00:25.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065483+0000", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1579999999999999e-05 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.178239+0000", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11275655800000001 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.179046+0000", 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00080670700000000004 2024-09-15T21:00:25.103 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs7 33/28 ECSubWrite(tid=31, reqid=client.4641.0:118, at_version=33'5, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.263707+0000", 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.82421117499999996, 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101239634, 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263707+0000", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263707+0000", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263708+0000", 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.511e-06 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263886+0000", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000177403 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263909+0000", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3116999999999999e-05 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263921+0000", 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2170999999999999e-05 2024-09-15T21:00:25.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.264114+0000", 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019338100000000001 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.264149+0000", 2024-09-15T21:00:25.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5101999999999997e-05 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.364852+0000", 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100702659 2024-09-15T21:00:25.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.364946+0000", 2024-09-15T21:00:25.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4289999999999993e-05 2024-09-15T21:00:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.112 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs7 33/28 ECSubWrite(tid=32, reqid=client.4641.0:119, at_version=33'6, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:25.112 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.263949+0000", 2024-09-15T21:00:25.112 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.82396881399999999, 2024-09-15T21:00:25.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102038249, 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263949+0000", 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263949+0000", 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263950+0000", 2024-09-15T21:00:25.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9999999999999998e-07 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263966+0000", 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5957999999999999e-05 2024-09-15T21:00:25.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263978+0000", 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2857000000000001e-05 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263986+0000", 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8890000000000002e-06 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.364329+0000", 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10034256599999999 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.364356+0000", 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7467e-05 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.365892+0000", 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001535872 2024-09-15T21:00:25.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.365987+0000", 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5140000000000001e-05 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:25.122 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.80754415499999999, 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20052589700000001, 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280375+0000", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.567e-06 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280390+0000", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4800000000000001e-05 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280411+0000", 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1009000000000002e-05 2024-09-15T21:00:25.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280423+0000", 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1717e-05 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480839+0000", 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20041608799999999 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480900+0000", 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0715999999999999e-05 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.80746785200000004, 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:00:25.127 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:25.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:00:25.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:25.131 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.80573735999999996, 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:00:25.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:00:25.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.135 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.80448978199999999, 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.79060567800000003, 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:00:25.141 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:25.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.79054757399999998, 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.183711915, 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297371+0000", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5999999999999999e-07 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297374+0000", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.213e-06 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297382+0000", 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5549999999999997e-06 2024-09-15T21:00:25.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297387+0000", 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9599999999999999e-06 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481022+0000", 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.18363590399999999 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481082+0000", 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9823e-05 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/28 ECSubWrite(tid=6, reqid=client.4641.0:109, at_version=33'5, trim_to=0'0, roll_forward_to=32'2))", 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.374365+0000", 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.71355332400000004, 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10247513599999999, 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.150 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374365+0000", 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374365+0000", 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374366+0000", 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5859999999999999e-06 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374640+0000", 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027349500000000001 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374685+0000", 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.545e-05 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474781+0000", 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100096239 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474938+0000", 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000156976 2024-09-15T21:00:25.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474964+0000", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5265999999999998e-05 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.476777+0000", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018132249999999999 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.476840+0000", 2024-09-15T21:00:25.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2898999999999995e-05 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70680423199999998, 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20016901200000001, 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381115+0000", 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4559999999999999e-06 2024-09-15T21:00:25.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381122+0000", 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381146+0000", 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3731999999999999e-05 2024-09-15T21:00:25.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381155+0000", 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0340000000000002e-06 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581244+0000", 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20008901500000001 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581283+0000", 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8884000000000002e-05 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70672367800000002, 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:25.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:00:25.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 33/28 ECSubWrite(tid=28, reqid=client.4641.0:122, at_version=33'10, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.474073+0000", 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.61384455599999999, 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101122347, 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474073+0000", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474073+0000", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474076+0000", 2024-09-15T21:00:25.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2970000000000002e-06 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474111+0000", 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4873000000000003e-05 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474143+0000", 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2305999999999999e-05 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474163+0000", 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0403000000000001e-05 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.574318+0000", 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100155249 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.574324+0000", 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.728e-06 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.575154+0000", 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00082950500000000002 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.575196+0000", 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1986000000000001e-05 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.167 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.61284532300000005, 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10645436699999999, 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475074+0000", 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.164e-06 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475083+0000", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3999999999999998e-06 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475108+0000", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4803e-05 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475125+0000", 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6690000000000001e-05 2024-09-15T21:00:25.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581408+0000", 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106283298 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581527+0000", 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011901200000000001 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.60427673000000004, 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:00:25.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:00:25.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:00:25.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.60335783200000004, 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:00:25.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:00:25.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.60083456300000004, 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:25.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:00:25.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:00:25.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:00:25.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.59853752500000001, 2024-09-15T21:00:25.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:00:25.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:00:25.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:00:25.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.190 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.190 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.190 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.190 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.190 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:36: finishing write tid 2 to smithi02732947-36 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:38: finishing write tid 1 to smithi02732947-38 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:48: finishing write tid 1 to smithi02732947-48 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:48: finishing write tid 2 to smithi02732947-48 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:43: finishing write tid 3 to smithi02732947-43 2024-09-15T21:00:25.193 INFO:tasks.rados.rados.0.smithi027.stdout:43: oid 43 updating version 0 to 3 2024-09-15T21:00:25.194 INFO:tasks.rados.rados.0.smithi027.stdout:43: oid 43 updating version 3 to 4 2024-09-15T21:00:25.194 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 43 v 4 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T21:00:25.194 INFO:tasks.rados.rados.0.smithi027.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-15T21:00:25.194 INFO:tasks.rados.rados.0.smithi027.stdout:46: finishing write tid 1 to smithi02732947-46 2024-09-15T21:00:25.194 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:25.196 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:25.189+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:25.196 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:25.193+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:25.203 INFO:tasks.rados.rados.0.smithi027.stdout:47: finishing write tid 1 to smithi02732947-47 2024-09-15T21:00:25.204 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.201+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:25.205 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.201+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:25.205 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.201+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:25.205 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.205+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:00:25.291 INFO:tasks.rados.rados.0.smithi027.stdout:48: finishing write tid 3 to smithi02732947-48 2024-09-15T21:00:25.292 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.286+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:25.292 INFO:tasks.rados.rados.0.smithi027.stdout:46: finishing write tid 2 to smithi02732947-46 2024-09-15T21:00:25.293 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.286+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:00:25.293 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.290+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,6 to read 0,1,4,5 2024-09-15T21:00:25.293 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:25.289+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:00:25.293 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:25.293+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:00:25.294 INFO:tasks.rados.rados.0.smithi027.stdout:42: finishing write tid 1 to smithi02732947-42 2024-09-15T21:00:25.294 INFO:tasks.rados.rados.0.smithi027.stdout:42: finishing write tid 2 to smithi02732947-42 2024-09-15T21:00:25.295 INFO:tasks.rados.rados.0.smithi027.stdout:44: finishing write tid 1 to smithi02732947-44 2024-09-15T21:00:25.295 INFO:tasks.rados.rados.0.smithi027.stdout:44: finishing write tid 2 to smithi02732947-44 2024-09-15T21:00:25.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.291+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:25.306 INFO:tasks.rados.rados.0.smithi027.stdout:45: finishing write tid 3 to smithi02732947-45 2024-09-15T21:00:25.307 INFO:tasks.rados.rados.0.smithi027.stdout:45: oid 45 updating version 0 to 6 2024-09-15T21:00:25.307 INFO:tasks.rados.rados.0.smithi027.stdout:45: oid 45 version 6 is already newer than 5 2024-09-15T21:00:25.307 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 45 v 6 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-15T21:00:25.307 INFO:tasks.rados.rados.0.smithi027.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-15T21:00:25.308 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.302+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:25.308 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.302+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:25.308 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.302+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:00:25.308 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.302+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:00:25.308 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:25.305+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,7 to read 0,1,4,5 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:48: finishing write tid 4 to smithi02732947-48 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:48: oid 48 updating version 0 to 4 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:48: oid 48 version 4 is already newer than 3 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:48: oid 48 updating version 4 to 5 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 48 v 5 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-15T21:00:25.309 INFO:tasks.rados.rados.0.smithi027.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-15T21:00:25.330 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:25.330 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:25.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.330 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.9s2 33/28 ECSubWrite(tid=14, reqid=client.4641.0:140, at_version=33'8, trim_to=0'0, roll_forward_to=33'7))", 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:25.307408+0000", 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.022039645, 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.022078674999999999, 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.307408+0000", 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.307408+0000", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.307410+0000", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.307441+0000", 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0936000000000003e-05 2024-09-15T21:00:25.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.307470+0000", 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9232999999999999e-05 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:00:25.333 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:25.345 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:25.383 INFO:tasks.rados.rados.0.smithi027.stdout:38: finishing write tid 2 to smithi02732947-38 2024-09-15T21:00:25.385 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.378+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:25.385 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.378+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:25.385 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.382+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:25.385 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.382+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:25.386 INFO:tasks.rados.rados.0.smithi027.stdout:36: finishing write tid 3 to smithi02732947-36 2024-09-15T21:00:25.386 INFO:tasks.rados.rados.0.smithi027.stdout:36: oid 36 updating version 0 to 12 2024-09-15T21:00:25.386 INFO:tasks.rados.rados.0.smithi027.stdout:36: oid 36 version 12 is already newer than 11 2024-09-15T21:00:25.386 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 36 v 12 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T21:00:25.386 INFO:tasks.rados.rados.0.smithi027.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-15T21:00:25.392 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.386+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:25.393 INFO:tasks.rados.rados.0.smithi027.stdout:49: finishing write tid 1 to smithi02732947-49 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:49: finishing write tid 2 to smithi02732947-49 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:49: finishing write tid 3 to smithi02732947-49 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:39: finishing write tid 3 to smithi02732947-39 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:39: oid 39 updating version 0 to 11 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:39: oid 39 updating version 11 to 12 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T21:00:25.394 INFO:tasks.rados.rados.0.smithi027.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-15T21:00:25.397 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.391+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:25.414 INFO:tasks.rados.rados.0.smithi027.stdout:51: finishing copy_from racing read to smithi02732947-2 2024-09-15T21:00:25.473 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:25.473 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:25.473 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:25.473 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:25.473 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:25.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:25.495 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.490+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:25.496 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.490+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:25.496 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.490+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:25.496 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.490+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:00:25.496 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.490+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,7 to read 0,1,4,5 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:47: finishing write tid 2 to smithi02732947-47 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:49: finishing write tid 4 to smithi02732947-49 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:49: oid 49 updating version 0 to 9 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:49: oid 49 version 9 is already newer than 8 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:49: oid 49 version 9 is already newer than 7 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T21:00:25.497 INFO:tasks.rados.rados.0.smithi027.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-15T21:00:25.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.491+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:25.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.491+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:00:25.498 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:25.495+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:25.499 INFO:tasks.rados.rados.0.smithi027.stdout:41: finishing write tid 3 to smithi02732947-41 2024-09-15T21:00:25.499 INFO:tasks.rados.rados.0.smithi027.stdout:41: oid 41 updating version 0 to 9 2024-09-15T21:00:25.499 INFO:tasks.rados.rados.0.smithi027.stdout:41: oid 41 updating version 9 to 10 2024-09-15T21:00:25.499 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-15T21:00:25.499 INFO:tasks.rados.rados.0.smithi027.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-15T21:00:25.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.495+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:25.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.495+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:25.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.495+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:25.588 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.586+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:25.588 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.586+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:25.588 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.586+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:25.588 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:25.586+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6 to read 0,1,4,5 2024-09-15T21:00:25.588 INFO:tasks.rados.rados.0.smithi027.stdout:38: finishing write tid 3 to smithi02732947-38 2024-09-15T21:00:25.588 INFO:tasks.rados.rados.0.smithi027.stdout:38: oid 38 updating version 0 to 13 2024-09-15T21:00:25.589 INFO:tasks.rados.rados.0.smithi027.stdout:38: oid 38 updating version 13 to 14 2024-09-15T21:00:25.589 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 14 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-15T21:00:25.589 INFO:tasks.rados.rados.0.smithi027.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-15T21:00:25.594 INFO:tasks.rados.rados.0.smithi027.stdout:51: finishing copy_from to smithi02732947-2 2024-09-15T21:00:25.594 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 2 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T21:00:25.594 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:25.587+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:25.597 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.590+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:25.597 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.590+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:00:25.597 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:25.594+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:46: finishing write tid 3 to smithi02732947-46 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:46: oid 46 updating version 0 to 13 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:46: oid 46 updating version 13 to 14 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 46 v 14 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:42: finishing write tid 3 to smithi02732947-42 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:42: oid 42 updating version 0 to 12 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:42: oid 42 version 12 is already newer than 11 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T21:00:25.601 INFO:tasks.rados.rados.0.smithi027.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-15T21:00:25.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.595+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:25.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.595+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:25.617 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 28.155328451999999, 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33373998100000002, 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457904+0000", 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457918+0000", 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4093999999999999e-05 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.620 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457944+0000", 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5986000000000002e-05 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457965+0000", 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0954e-05 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458117+0000", 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015210500000000001 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458129+0000", 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.149e-05 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539011+0000", 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.080882495999999998 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539035+0000", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3439999999999999e-05 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788238+0000", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.24920330900000001 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788633+0000", 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039541700000000001 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.789053+0000", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00041907000000000001 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791200+0000", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021477969999999999 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791447+0000", 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024629799999999999 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791572+0000", 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012488899999999999 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791641+0000", 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9843000000000002e-05 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.994979587, 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:00:25.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:00:25.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:00:25.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:00:25.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.381871087, 2024-09-15T21:00:25.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30544877100000001, 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 23 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231359+0000", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231360+0000", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1000000000000001e-06 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231495+0000", 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000134659 2024-09-15T21:00:25.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231512+0000", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7297000000000001e-05 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231521+0000", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5959999999999993e-06 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234389+0000", 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002867152 2024-09-15T21:00:25.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234510+0000", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121796 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.234715+0000", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000204191 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333094+0000", 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.098379878000000004 2024-09-15T21:00:25.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536645+0000", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203550593 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536729+0000", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3880000000000003e-05 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.536808+0000", 2024-09-15T21:00:25.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8628999999999995e-05 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.381685328, 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.637 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.639 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.640 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:00:25.641 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:00:25.642 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.643 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.644 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.381500626000001, 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.645 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:25.646 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.647 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.176224614999999, 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.652 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:25.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:00:25.658 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.175899454, 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.659 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:00:25.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.664 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.175858597000001, 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.665 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.670 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.175809851, 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.672 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:00:25.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.17549494, 2024-09-15T21:00:25.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:00:25.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:00:25.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:00:25.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.175416874, 2024-09-15T21:00:25.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:00:25.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.480958557000001, 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31281666299999999, 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.691 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 71 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132272+0000", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132278+0000", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7570000000000001e-06 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132291+0000", 2024-09-15T21:00:25.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2703e-05 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132299+0000", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0069999999999997e-06 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139452+0000", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071530079999999998 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139562+0000", 2024-09-15T21:00:25.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000110432 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139743+0000", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018057899999999999 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142717+0000", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002974661 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444884+0000", 2024-09-15T21:00:25.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216663399999999 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444969+0000", 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4452999999999999e-05 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445088+0000", 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119449 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.480917924, 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31312449999999997, 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 72 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.697 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132313+0000", 2024-09-15T21:00:25.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132422+0000", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109033 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132439+0000", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7586000000000001e-05 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132449+0000", 2024-09-15T21:00:25.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6199999999999994e-06 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140318+0000", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078687559999999993 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140430+0000", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112589 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140615+0000", 2024-09-15T21:00:25.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018495600000000001 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142725+0000", 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021100300000000002 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445273+0000", 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30254736500000001 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445366+0000", 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3249000000000001e-05 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445436+0000", 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0406000000000003e-05 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:25.702 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.480762300999999, 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31325695799999997, 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 73 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132469+0000", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132475+0000", 2024-09-15T21:00:25.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4999999999999996e-06 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132493+0000", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7669999999999999e-05 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132507+0000", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3966e-05 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141793+0000", 2024-09-15T21:00:25.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092860819999999993 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141919+0000", 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000126396 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142125+0000", 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020598799999999999 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.143405+0000", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012801570000000001 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445573+0000", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216763699999999 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445651+0000", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8239999999999996e-05 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445725+0000", 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3329000000000002e-05 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.480677516, 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31343764099999999, 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.708 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 74 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132554+0000", 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.327e-06 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132565+0000", 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0967e-05 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132584+0000", 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.984e-05 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232738+0000", 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10015367 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232890+0000", 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015175200000000001 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233152+0000", 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026189099999999999 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233532+0000", 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038017699999999999 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239440+0000", 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005908122 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445839+0000", 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20639856300000001 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445917+0000", 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8055999999999999e-05 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445990+0000", 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3275999999999994e-05 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:25.714 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 23.380187005, 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31253853100000001, 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 76 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.715 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.716 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233045+0000", 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233061+0000", 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6453e-05 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233105+0000", 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4063000000000001e-05 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233123+0000", 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7937000000000001e-05 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239508+0000", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063845940000000004 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239647+0000", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013887599999999999 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239834+0000", 2024-09-15T21:00:25.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018760200000000001 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.243229+0000", 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033952729999999999 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545355+0000", 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30212512200000002 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545446+0000", 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1941999999999995e-05 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545581+0000", 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013493199999999999 2024-09-15T21:00:25.721 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:104 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 22.301250627999998, 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30628519900000001, 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.722 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311980+0000", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312116+0000", 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000136129 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312137+0000", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1026000000000002e-05 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312147+0000", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.594e-06 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316046+0000", 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003899062 2024-09-15T21:00:25.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316173+0000", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012724000000000001 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316430+0000", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025629800000000002 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.317718+0000", 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287949 2024-09-15T21:00:25.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618117+0000", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30039965099999999 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618203+0000", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5415999999999997e-05 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618264+0000", 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1820000000000001e-05 2024-09-15T21:00:25.727 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.385033608000001, 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30562285500000003, 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.728 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 116 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228196+0000", 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228198+0000", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.407e-06 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228209+0000", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0993e-05 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228225+0000", 2024-09-15T21:00:25.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6469000000000001e-05 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228241+0000", 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5739999999999998e-05 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.338640+0000", 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11039951200000001 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.338852+0000", 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000212062 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.339048+0000", 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019553100000000001 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343126+0000", 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0040778849999999998 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533696+0000", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19057063299999999 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533767+0000", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0049000000000006e-05 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.533819+0000", 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2574000000000002e-05 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.733 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.384961169, 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:00:25.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:25.739 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.384791126, 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:00:25.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:00:25.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:25.745 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:25.746 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:25.747 INFO:tasks.rados.rados.0.smithi027.stdout:47: finishing write tid 3 to smithi02732947-47 2024-09-15T21:00:25.747 INFO:tasks.rados.rados.0.smithi027.stdout:47: oid 47 updating version 0 to 7 2024-09-15T21:00:25.747 INFO:tasks.rados.rados.0.smithi027.stdout:47: oid 47 updating version 7 to 8 2024-09-15T21:00:25.747 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 47 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:00:25.747 INFO:tasks.rados.rados.0.smithi027.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-15T21:00:25.748 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.697+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:25.748 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.697+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:25.748 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.697+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,7 to read 0,1,4,5 2024-09-15T21:00:25.748 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:25.697+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:00:25.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:25.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.695+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:25.749 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.742+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:25.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:25.795+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:25.801 INFO:tasks.rados.rados.0.smithi027.stdout:44: finishing write tid 3 to smithi02732947-44 2024-09-15T21:00:25.802 INFO:tasks.rados.rados.0.smithi027.stdout:44: oid 44 updating version 0 to 13 2024-09-15T21:00:25.802 INFO:tasks.rados.rados.0.smithi027.stdout:44: oid 44 updating version 13 to 14 2024-09-15T21:00:25.802 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 44 v 14 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T21:00:25.802 INFO:tasks.rados.rados.0.smithi027.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:25.849 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:25.847+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:25.880 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:25.880 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:25.880 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:25.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:36: done (15 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:38: done (14 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:39: done (13 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:41: done (12 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:42: done (11 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:43: done (10 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:44: done (9 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:45: done (8 left) 2024-09-15T21:00:25.903 INFO:tasks.rados.rados.0.smithi027.stdout:46: done (7 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:47: done (6 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:48: done (5 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:49: done (4 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:51: done (3 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:54: done (2 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:55: read oid 11 snap -1 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:55: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:56: copy_from oid 28 from oid 46 current snap is 0 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:57: rollback oid 44 current snap is 0 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:57: done (4 left) 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:58: copy_from oid 22 from oid 47 current snap is 0 2024-09-15T21:00:25.904 INFO:tasks.rados.rados.0.smithi027.stdout:59: delete oid 1 current snap is 0 2024-09-15T21:00:25.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:25.902+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:25.949 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:25.949+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:26.005 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.002+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:26.008 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.005+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:26.008 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.002+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6,7 to read 0,1,4,5 2024-09-15T21:00:26.009 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.005+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:00:26.009 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.009+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:00:26.011 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.009+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,4,5 2024-09-15T21:00:26.011 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:26.006+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:26.012 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:26.013 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:26.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:26.036 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:00:26.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.047+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:26.109 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.102+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:26.110 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.106+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.111 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.106+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.111 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.106+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:26.111 INFO:tasks.rados.rados.0.smithi027.stdout:59: done (5 left) 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:60: read oid 17 snap -1 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:60: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:61: read oid 29 snap -1 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:61: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:62: read oid 16 snap -1 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:62: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T21:00:26.112 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 6 current snap is 0 2024-09-15T21:00:26.113 INFO:tasks.rados.rados.0.smithi027.stdout:63: seq_num 53 ranges {98304=524288,622592=376832} 2024-09-15T21:00:26.113 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.106+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:26.117 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.110+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.117 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.114+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.117 INFO:tasks.rados.rados.0.smithi027.stdout:63: writing smithi02732947-6 from 98304 to 622592 tid 1 2024-09-15T21:00:26.118 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.114+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:26.118 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.114+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:26.120 INFO:tasks.rados.rados.0.smithi027.stdout:63: writing smithi02732947-6 from 622592 to 999424 tid 2 2024-09-15T21:00:26.121 INFO:tasks.rados.rados.0.smithi027.stdout:64: copy_from oid 23 from oid 45 current snap is 0 2024-09-15T21:00:26.121 INFO:tasks.rados.rados.0.smithi027.stdout:58: finishing copy_from racing read to smithi02732947-22 2024-09-15T21:00:26.121 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 4 current snap is 0 2024-09-15T21:00:26.121 INFO:tasks.rados.rados.0.smithi027.stdout:65: seq_num 54 ranges {65536=540672,606208=376832} 2024-09-15T21:00:26.125 INFO:tasks.rados.rados.0.smithi027.stdout:65: writing smithi02732947-4 from 65536 to 606208 tid 1 2024-09-15T21:00:26.128 INFO:tasks.rados.rados.0.smithi027.stdout:65: writing smithi02732947-4 from 606208 to 983040 tid 2 2024-09-15T21:00:26.128 INFO:tasks.rados.rados.0.smithi027.stdout:66: setattr oid 20 current snap is 0 2024-09-15T21:00:26.130 INFO:tasks.rados.rados.0.smithi027.stdout:67: read oid 27 snap -1 2024-09-15T21:00:26.130 INFO:tasks.rados.rados.0.smithi027.stdout:67: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T21:00:26.130 INFO:tasks.rados.rados.0.smithi027.stdout:68: delete oid 7 current snap is 0 2024-09-15T21:00:26.151 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.146+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:26.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:26.200 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.9062604990000001, 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:26.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:00:26.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:00:26.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.904957588, 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:00:26.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:00:26.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:00:26.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.9048776630000002, 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6983938140000001, 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:26.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.6969963689999998, 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.696957142, 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:00:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:00:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:00:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4951066740000001, 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4947358579999999, 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:00:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4945661960000001, 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:00:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4941580059999999, 2024-09-15T21:00:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:00:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:00:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4268781619999999, 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:00:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:00:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=10, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.4257227379999999, 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23004227399999999, 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772835+0000", 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5230000000000001e-06 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772844+0000", 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4670000000000004e-06 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772869+0000", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4955000000000001e-05 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873000+0000", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100131232 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.002843+0000", 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.129842652 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.002876+0000", 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3445000000000003e-05 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3956652599999999, 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:00:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=10, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3945418199999999, 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29916498600000002, 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804017+0000", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804029+0000", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2016e-05 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804059+0000", 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890999999999999e-05 2024-09-15T21:00:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804086+0000", 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7637999999999999e-05 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103155+0000", 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29906852699999997 2024-09-15T21:00:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103180+0000", 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981e-05 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3938963900000001, 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.391340534, 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295896192, 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807218+0000", 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9360000000000002e-06 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807227+0000", 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807254+0000", 2024-09-15T21:00:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7260999999999999e-05 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807268+0000", 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4005999999999999e-05 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103079+0000", 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29581095699999999 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103112+0000", 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3012000000000002e-05 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.32551066, 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23024503599999999, 2024-09-15T21:00:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873047+0000", 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0559999999999999e-06 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873054+0000", 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873072+0000", 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7183000000000001e-05 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873084+0000", 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2645e-05 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103254+0000", 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23017025499999999 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103291+0000", 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6414000000000001e-05 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.3252697630000001, 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.333051763, 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873288+0000", 2024-09-15T21:00:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873309+0000", 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1588000000000001e-05 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873333+0000", 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3842e-05 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873347+0000", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3799e-05 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206159+0000", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.33281165400000001 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206339+0000", 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000179803 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=10, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.986109+0000", 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.2124473060000001, 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.217375547, 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986109+0000", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986109+0000", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986111+0000", 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1789999999999998e-06 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986121+0000", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6860000000000001e-06 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986151+0000", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9981999999999999e-05 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986169+0000", 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8379000000000001e-05 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203427+0000", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.217257852 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203485+0000", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7469000000000002e-05 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.986198+0000", 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.2123581059999999, 2024-09-15T21:00:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.220352572, 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986198+0000", 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986198+0000", 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986199+0000", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9400000000000005e-07 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986217+0000", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7504999999999999e-05 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986231+0000", 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4263e-05 2024-09-15T21:00:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.986239+0000", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1920000000000005e-06 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206427+0000", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22018848899999999 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206551+0000", 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012352899999999999 2024-09-15T21:00:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:26.295 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:26.296 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:26.206+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:26.296 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:26.206+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:26.297 INFO:tasks.rados.rados.0.smithi027.stdout:55: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T21:00:26.297 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:26.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:26.298 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.213+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.213+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:26.217+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.210+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.210+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.210+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:26.299 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.210+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.210+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.214+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.214+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.214+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.214+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.214+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:26.300 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 163964 2024-09-15T21:00:26.300 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 51284 2024-09-15T21:00:26.301 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:26.290+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:26.318 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.314+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:00:26.324 INFO:tasks.rados.rados.0.smithi027.stdout:64: finishing copy_from racing read to smithi02732947-23 2024-09-15T21:00:26.350 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:26.351 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:26.367 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:26.367 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:26.383 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:26.384 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:26.397 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.391+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:26.400 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:26.401 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:26.421 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.414+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:26.421 INFO:tasks.rados.rados.0.smithi027.stdout:56: finishing copy_from racing read to smithi02732947-28 2024-09-15T21:00:26.421 INFO:tasks.rados.rados.0.smithi027.stdout:61: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-15T21:00:26.422 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:26.422 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:26.423 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:26.432 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:26.433 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:26.433 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:26.437 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 221975 2024-09-15T21:00:26.444 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:26.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:26.498 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:26.497+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:26.524 INFO:tasks.rados.rados.0.smithi027.stdout:56: finishing copy_from to smithi02732947-28 2024-09-15T21:00:26.525 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 28 v 14 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T21:00:26.545 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:26.545 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:26.571 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:26.571 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:26.571 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:26.571 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:26.571 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:26.574 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:26.574 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:26.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:26.602 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:26.616 INFO:tasks.rados.rados.0.smithi027.stdout:55: done (13 left) 2024-09-15T21:00:26.616 INFO:tasks.rados.rados.0.smithi027.stdout:56: done (12 left) 2024-09-15T21:00:26.616 INFO:tasks.rados.rados.0.smithi027.stdout:61: done (11 left) 2024-09-15T21:00:26.617 INFO:tasks.rados.rados.0.smithi027.stdout:68: done (10 left) 2024-09-15T21:00:26.617 INFO:tasks.rados.rados.0.smithi027.stdout:69: read oid 29 snap -1 2024-09-15T21:00:26.617 INFO:tasks.rados.rados.0.smithi027.stdout:69: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-15T21:00:26.617 INFO:tasks.rados.rados.0.smithi027.stdout:70: setattr oid 15 current snap is 0 2024-09-15T21:00:26.618 INFO:tasks.rados.rados.0.smithi027.stdout:50: finishing write tid 1 to smithi02732947-50 2024-09-15T21:00:26.618 INFO:tasks.rados.rados.0.smithi027.stdout:50: finishing write tid 2 to smithi02732947-50 2024-09-15T21:00:26.618 INFO:tasks.rados.rados.0.smithi027.stdout:50: finishing write tid 3 to smithi02732947-50 2024-09-15T21:00:26.618 INFO:tasks.rados.rados.0.smithi027.stdout:71: snap_create 2024-09-15T21:00:26.619 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.614+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:26.619 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.614+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:26.619 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.614+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:26.619 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.614+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.620 INFO:tasks.rados.rados.0.smithi027.stdout:58: finishing copy_from to smithi02732947-22 2024-09-15T21:00:26.621 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:00:26.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.615+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:26.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.615+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:00:26.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.619+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,6 to read 0,1,4,5 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:50: finishing write tid 4 to smithi02732947-50 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:50: oid 50 updating version 0 to 17 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:50: oid 50 version 17 is already newer than 15 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:50: oid 50 version 17 is already newer than 16 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 50 v 17 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-15T21:00:26.625 INFO:tasks.rados.rados.0.smithi027.stdout:52: finishing write tid 1 to smithi02732947-25 2024-09-15T21:00:26.629 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:26.629+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:26.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:26.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.631+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.631+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.631+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.631+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.631+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.660 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:26.661 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:26.684 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:26.684 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:26.699 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.694+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:26.721 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.714+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:26.721 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.718+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:26.722 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.718+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:26.723 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.718+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:26.723 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.718+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:26.727 INFO:tasks.rados.rados.0.smithi027.stdout:69: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-15T21:00:26.735 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 461532 2024-09-15T21:00:26.736 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 96150 2024-09-15T21:00:26.746 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:26.750 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:26.750 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:26.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:26.800 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:26.798+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:26.823 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 20 v 7 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T21:00:26.848 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:26.849 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:26.887 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:26.887 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:26.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:26.902 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.899+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:26.923 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:26.929 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:26.930 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:26.923+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,7 to read 0,1,4,5 2024-09-15T21:00:27.002 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:26.998+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:27.023 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.023 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:63: finishing write tid 1 to smithi02732947-6 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:63: finishing write tid 2 to smithi02732947-6 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:63: finishing write tid 3 to smithi02732947-6 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:65: finishing write tid 1 to smithi02732947-4 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:65: finishing write tid 2 to smithi02732947-4 2024-09-15T21:00:27.042 INFO:tasks.rados.rados.0.smithi027.stdout:65: finishing write tid 3 to smithi02732947-4 2024-09-15T21:00:27.046 INFO:tasks.rados.rados.0.smithi027.stdout:50: done (12 left) 2024-09-15T21:00:27.046 INFO:tasks.rados.rados.0.smithi027.stdout:58: done (11 left) 2024-09-15T21:00:27.046 INFO:tasks.rados.rados.0.smithi027.stdout:66: done (10 left) 2024-09-15T21:00:27.046 INFO:tasks.rados.rados.0.smithi027.stdout:69: done (9 left) 2024-09-15T21:00:27.047 INFO:tasks.rados.rados.0.smithi027.stdout:70: done (8 left) 2024-09-15T21:00:27.047 INFO:tasks.rados.rados.0.smithi027.stdout:71: done (7 left) 2024-09-15T21:00:27.047 INFO:tasks.rados.rados.0.smithi027.stdout:72: rollback oid 30 current snap is 1 2024-09-15T21:00:27.047 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 30 to 0 2024-09-15T21:00:27.047 INFO:tasks.rados.rados.0.smithi027.stdout:73: snap_create 2024-09-15T21:00:27.103 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.099+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:27.123 INFO:tasks.rados.rados.0.smithi027.stdout:64: finishing copy_from to smithi02732947-23 2024-09-15T21:00:27.124 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 23 v 6 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-15T21:00:27.156 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.156 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:27.205 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:27.198+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:27.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.223+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.223+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.227+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.227+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.243 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.241+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:27.289 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.290 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:27.305 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:27.301+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:27.335 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.327+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.344 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.341+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:27.344 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.341+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:27.344 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.341+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:27.344 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.341+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,4,5 2024-09-15T21:00:27.344 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:27.341+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,4,5 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:63: finishing write tid 4 to smithi02732947-6 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:63: oid 6 updating version 0 to 6 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:63: oid 6 updating version 6 to 7 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:63: oid 6 version 7 is already newer than 5 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 7 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:63: left oid 6 (ObjNum 53 snap 0 seq_num 53) 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:65: finishing write tid 4 to smithi02732947-4 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:65: oid 4 updating version 0 to 10 2024-09-15T21:00:27.345 INFO:tasks.rados.rados.0.smithi027.stdout:65: oid 4 version 10 is already newer than 8 2024-09-15T21:00:27.346 INFO:tasks.rados.rados.0.smithi027.stdout:65: oid 4 version 10 is already newer than 9 2024-09-15T21:00:27.346 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 10 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-15T21:00:27.346 INFO:tasks.rados.rados.0.smithi027.stdout:65: left oid 4 (ObjNum 54 snap 0 seq_num 54) 2024-09-15T21:00:27.406 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:27.406+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:27.427 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.427 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.427+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.427+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:27.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.427+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:27.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.427+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:27.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:27.507 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:27.503+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:27.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.527+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.533 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.527+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:27.535 INFO:tasks.rados.rados.0.smithi027.stdout:72: finishing rollback tid 0 to smithi02732947-30 2024-09-15T21:00:27.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.531+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:27.539 INFO:tasks.rados.rados.0.smithi027.stdout:60: expect (ObjNum 16 snap 0 seq_num 16) 2024-09-15T21:00:27.539 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:27.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:27.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:27.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:27.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:27.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:27.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,4,5 2024-09-15T21:00:27.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.535+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,4,5 2024-09-15T21:00:27.546 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:27.553 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:27.602 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.603 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.608 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:27.603+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:27.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:27.636 INFO:tasks.rados.rados.0.smithi027.stdout:72: finishing rollback tid 1 to smithi02732947-30 2024-09-15T21:00:27.636 INFO:tasks.rados.rados.0.smithi027.stdout:72: finishing rollback tid 2 to smithi02732947-30 2024-09-15T21:00:27.636 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 19 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T21:00:27.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.635+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:27.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.635+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:27.643 INFO:tasks.rados.rados.0.smithi027.stdout:62: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-15T21:00:27.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.639+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:00:27.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:27.639+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:00:27.645 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 232626 2024-09-15T21:00:27.647 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 254150 2024-09-15T21:00:27.648 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:27.651 INFO:tasks.rados.rados.0.smithi027.stdout:67: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T21:00:27.652 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 198718 2024-09-15T21:00:27.656 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 49729 2024-09-15T21:00:27.661 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:27.666 INFO:tasks.rados.rados.0.smithi027.stdout:52: finishing write tid 2 to smithi02732947-25 2024-09-15T21:00:27.709 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:27.703+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:27.756 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.756 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:27.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:27.803+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:27.910 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:27.907+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:27.940 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:27.940 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:27.950 INFO:tasks.rados.rados.0.smithi027.stdout:52: finishing write tid 3 to smithi02732947-25 2024-09-15T21:00:27.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:28.011 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:28.006+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:60: done (8 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:62: done (7 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:63: done (6 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:64: done (5 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:65: done (4 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:67: done (3 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:72: done (2 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:73: done (1 left) 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:74: rollback oid 37 current snap is 2 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 37 to 0 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:75: read oid 6 snap 1 2024-09-15T21:00:28.052 INFO:tasks.rados.rados.0.smithi027.stdout:75: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-15T21:00:28.053 INFO:tasks.rados.rados.0.smithi027.stdout:76: read oid 47 snap -1 2024-09-15T21:00:28.053 INFO:tasks.rados.rados.0.smithi027.stdout:76: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T21:00:28.053 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 5 current snap is 2 2024-09-15T21:00:28.053 INFO:tasks.rados.rados.0.smithi027.stdout:77: seq_num 56 ranges {81920=573440,655360=376832} 2024-09-15T21:00:28.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:00:28.065 INFO:tasks.rados.rados.0.smithi027.stdout:77: writing smithi02732947-5 from 81920 to 655360 tid 1 2024-09-15T21:00:28.069 INFO:tasks.rados.rados.0.smithi027.stdout:77: writing smithi02732947-5 from 655360 to 1032192 tid 2 2024-09-15T21:00:28.069 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 16 current snap is 2 2024-09-15T21:00:28.069 INFO:tasks.rados.rados.0.smithi027.stdout:78: seq_num 57 ranges {262144=589824,851968=376832} 2024-09-15T21:00:28.082 INFO:tasks.rados.rados.0.smithi027.stdout:78: writing smithi02732947-16 from 262144 to 851968 tid 1 2024-09-15T21:00:28.084 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.084 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:78: writing smithi02732947-16 from 851968 to 1228800 tid 2 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:79: rollback oid 30 current snap is 2 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 30 to 0 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:80: rollback oid 27 current snap is 2 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 27 to 1 2024-09-15T21:00:28.089 INFO:tasks.rados.rados.0.smithi027.stdout:81: snap_create 2024-09-15T21:00:28.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:28.112 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:28.107+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:28.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:28.207+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:28.251 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:28.247+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:28.251 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:28.247+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:28.251 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:28.247+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:28.269 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.269 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:28.329 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:28.326+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:52: finishing write tid 4 to smithi02732947-25 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:52: oid 25 updating version 0 to 20 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:52: oid 25 version 20 is already newer than 18 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:52: oid 25 version 20 is already newer than 19 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 20 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T21:00:28.353 INFO:tasks.rados.rados.0.smithi027.stdout:52: left oid 25 (ObjNum 50 snap 0 seq_num 50) 2024-09-15T21:00:28.358 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.357+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:28.358 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.357+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:00:28.362 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.361+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:28.363 INFO:tasks.rados.rados.0.smithi027.stdout:77: finishing write tid 1 to smithi02732947-5 2024-09-15T21:00:28.363 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.361+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:28.363 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.361+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,4,5 2024-09-15T21:00:28.363 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.361+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:28.365 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.361+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:28.405 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.405 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.415 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:28.411+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:28.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:28.459 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.457+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:28.461 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.457+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:28.465 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.461+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:28.465 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.461+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:28.465 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.461+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:28.465 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.461+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,4,5 2024-09-15T21:00:28.465 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.465+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5 to read 0,1,4,5 2024-09-15T21:00:28.516 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:28.511+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:28.557 INFO:tasks.rados.rados.0.smithi027.stdout:74: finishing rollback tid 0 to smithi02732947-37 2024-09-15T21:00:28.557 INFO:tasks.rados.rados.0.smithi027.stdout:74: finishing rollback tid 1 to smithi02732947-37 2024-09-15T21:00:28.559 INFO:tasks.rados.rados.0.smithi027.stdout:77: finishing write tid 2 to smithi02732947-5 2024-09-15T21:00:28.571 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.569+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:28.571 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.569+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:28.571 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:28.569+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4 to read 0,1,4,5 2024-09-15T21:00:28.588 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.588 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.597 INFO:tasks.rados.rados.0.smithi027.stdout:79: finishing rollback tid 0 to smithi02732947-30 2024-09-15T21:00:28.598 INFO:tasks.rados.rados.0.smithi027.stdout:79: finishing rollback tid 1 to smithi02732947-30 2024-09-15T21:00:28.598 INFO:tasks.rados.rados.0.smithi027.stdout:79: finishing rollback tid 2 to smithi02732947-30 2024-09-15T21:00:28.598 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 23 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T21:00:28.604 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:28.649 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:28.646+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:28.659 INFO:tasks.rados.rados.0.smithi027.stdout:74: finishing rollback tid 2 to smithi02732947-37 2024-09-15T21:00:28.659 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 10 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T21:00:28.671 INFO:tasks.rados.rados.0.smithi027.stdout:78: finishing write tid 1 to smithi02732947-16 2024-09-15T21:00:28.671 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.669+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:28.672 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.669+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:00:28.672 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.669+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5 to read 0,1,4,5 2024-09-15T21:00:28.673 INFO:tasks.rados.rados.0.smithi027.stdout:75: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-15T21:00:28.674 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 129282 2024-09-15T21:00:28.685 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:28.699 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:28.719 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:28.715+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:28.730 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.730 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:28.774 INFO:tasks.rados.rados.0.smithi027.stdout:76: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T21:00:28.778 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 443194 2024-09-15T21:00:28.783 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:28.798 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 454097 2024-09-15T21:00:28.820 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.817+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:28.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:28.898 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:28.899 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:28.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:28.921 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:28.917+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:28.966 INFO:tasks.rados.rados.0.smithi027.stdout:77: finishing write tid 3 to smithi02732947-5 2024-09-15T21:00:29.021 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:29.018+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:29.022 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:29.023 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:29.039 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:29.047 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:00:29.049 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:29.049 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:29.050 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-15T21:00:29.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:52: done (8 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:74: done (7 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:75: done (6 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:76: done (5 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:79: done (4 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:81: done (3 left) 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 45 current snap is 3 2024-09-15T21:00:29.051 INFO:tasks.rados.rados.0.smithi027.stdout:82: seq_num 58 ranges {753664=606208,1359872=376832} 2024-09-15T21:00:29.058 INFO:tasks.rados.rados.0.smithi027.stdout:82: writing smithi02732947-45 from 753664 to 1359872 tid 1 2024-09-15T21:00:29.061 INFO:tasks.rados.rados.0.smithi027.stdout:82: writing smithi02732947-45 from 1359872 to 1736704 tid 2 2024-09-15T21:00:29.061 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 9 current snap is 3 2024-09-15T21:00:29.061 INFO:tasks.rados.rados.0.smithi027.stdout:83: seq_num 59 ranges {147456=622592,770048=376832} 2024-09-15T21:00:29.063 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:29.063 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:29.066 INFO:tasks.rados.rados.0.smithi027.stdout:83: writing smithi02732947-9 from 147456 to 770048 tid 1 2024-09-15T21:00:29.068 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.062+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:29.068 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.062+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:29.069 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.062+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:00:29.069 INFO:tasks.rados.rados.0.smithi027.stdout:83: writing smithi02732947-9 from 770048 to 1146880 tid 2 2024-09-15T21:00:29.069 INFO:tasks.rados.rados.0.smithi027.stdout:84: read oid 31 snap -1 2024-09-15T21:00:29.069 INFO:tasks.rados.rados.0.smithi027.stdout:84: expect (ObjNum 30 snap 0 seq_num 30) 2024-09-15T21:00:29.069 INFO:tasks.rados.rados.0.smithi027.stdout:85: snap_remove snap 0 2024-09-15T21:00:29.088 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:29.088 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:29.095 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_repop(client.4254.0:1 2.3 e22/21 2:c4c92e5a:::rbd_trash:head v 22'1, mlcod=0'0)", 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:06.194653+0000", 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.900449371000001, 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026411160000000002, 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194653+0000", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194653+0000", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194654+0000", 2024-09-15T21:00:29.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194661+0000", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5700000000000004e-06 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194683+0000", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.181e-05 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194699+0000", 2024-09-15T21:00:29.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.575e-05 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194820+0000", 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121489 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.194889+0000", 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9165999999999995e-05 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.197195+0000", 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002305288 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:06.197294+0000", 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9532999999999997e-05 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 32/28 ECSubWrite(tid=23, reqid=client.4641.0:71, at_version=0'0, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.762296+0000", 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.3328063779999999, 2024-09-15T21:00:29.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00157774, 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762296+0000", 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762296+0000", 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762299+0000", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762327+0000", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8697e-05 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762376+0000", 2024-09-15T21:00:29.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8937000000000003e-05 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762402+0000", 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5924000000000001e-05 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762583+0000", 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018061100000000001 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.762608+0000", 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5786999999999999e-05 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.763799+0000", 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011904649999999999 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.763874+0000", 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4878999999999999e-05 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 32/28 ECSubRead(tid=25, to_read={3:a5da2594:::smithi02732947-25:head=0,4096,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9317779929999999, 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10096388000000001, 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163325+0000", 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8199999999999999e-07 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163332+0000", 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1960000000000002e-06 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163354+0000", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2116e-05 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163372+0000", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8238000000000001e-05 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263745+0000", 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100372425 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264288+0000", 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054372299999999997 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:121, at_version=32'9, trim_to=0'0, roll_forward_to=32'7))", 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.267863+0000", 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8272394539999999, 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002157707, 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267863+0000", 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267863+0000", 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267865+0000", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3159999999999999e-06 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.268369+0000", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050346099999999997 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.268407+0000", 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7826e-05 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.268424+0000", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.268550+0000", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012562200000000001 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.268566+0000", 2024-09-15T21:00:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6359999999999999e-05 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.269966+0000", 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013996060000000001 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.270021+0000", 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5167999999999998e-05 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 33/28 ECSubRead(tid=15, to_read={3:2f703215:::smithi02732947-30:head=0,4096,0}, subchunks={3:2f703215:::smithi02732947-30:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8151873839999997, 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10090344699999999, 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279917+0000", 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.897e-06 2024-09-15T21:00:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279934+0000", 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7198e-05 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279967+0000", 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3197000000000001e-05 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279991+0000", 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3487000000000001e-05 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380280+0000", 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100288711 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380819+0000", 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053895699999999996 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8150536539999997, 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134782000000001, 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280050+0000", 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280059+0000", 2024-09-15T21:00:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4259999999999992e-06 2024-09-15T21:00:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280077+0000", 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7948000000000002e-05 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280091+0000", 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4338999999999999e-05 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380947+0000", 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100855271 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381397+0000", 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044987 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=17, reqid=client.4641.0:106, at_version=33'11, trim_to=0'0, roll_forward_to=32'10))", 2024-09-15T21:00:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8149664200000002, 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103549026, 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280137+0000", 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280251+0000", 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000113705 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280273+0000", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2198e-05 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280285+0000", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2014999999999999e-05 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381500+0000", 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10121572400000001 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381526+0000", 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5077000000000001e-05 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383596+0000", 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020703029999999999 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383685+0000", 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9223999999999998e-05 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=18, reqid=client.4641.0:106, at_version=0'0, trim_to=0'0, roll_forward_to=33'11))", 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.573383+0000", 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.5217195180000003, 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016439040000000001, 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573383+0000", 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573383+0000", 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573392+0000", 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3430000000000002e-06 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573429+0000", 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6585999999999998e-05 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573472+0000", 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2661999999999999e-05 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573533+0000", 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1970000000000005e-05 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573767+0000", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023397899999999999 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573799+0000", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1636e-05 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.574955+0000", 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011558989999999999 2024-09-15T21:00:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.575027+0000", 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1829000000000006e-05 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=29, reqid=client.4641.0:124, at_version=33'11, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.5177295830000004, 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103593675, 2024-09-15T21:00:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577375+0000", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9870000000000002e-06 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577857+0000", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048242599999999999 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577888+0000", 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1007000000000003e-05 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577904+0000", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5588000000000001e-05 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578065+0000", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000161593 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578111+0000", 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5977999999999999e-05 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680842+0000", 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102730619 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680967+0000", 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012447700000000001 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=30, reqid=client.4641.0:125, at_version=33'12, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.517158212, 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10310723500000001, 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577945+0000", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.096e-06 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577980+0000", 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4836000000000003e-05 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577999+0000", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8975e-05 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578040+0000", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1328999999999997e-05 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579237+0000", 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011965879999999999 2024-09-15T21:00:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579269+0000", 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.222e-05 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680983+0000", 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10171382499999999 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681051+0000", 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8366000000000002e-05 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=31, reqid=client.4641.0:130, at_version=33'13, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.5170309990000002, 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10373313300000001, 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578073+0000", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.319e-06 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578381+0000", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000308282 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578400+0000", 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8788e-05 2024-09-15T21:00:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578412+0000", 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2509e-05 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679722+0000", 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10130917 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679753+0000", 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1386000000000001e-05 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681744+0000", 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019913180000000002 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681805+0000", 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0361e-05 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=19, reqid=client.4641.0:107, at_version=33'12, trim_to=0'0, roll_forward_to=33'11))", 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.876789+0000", 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.2183130999999996, 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018434409999999999, 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876789+0000", 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876789+0000", 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876791+0000", 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876827+0000", 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5132999999999997e-05 2024-09-15T21:00:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876864+0000", 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7456000000000002e-05 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876889+0000", 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4890999999999999e-05 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877082+0000", 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019275299999999999 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877115+0000", 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3229999999999999e-05 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878532+0000", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014174979999999999 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878633+0000", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010032 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=20, reqid=client.4641.0:112, at_version=33'13, trim_to=0'0, roll_forward_to=33'11))", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.978739+0000", 2024-09-15T21:00:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.1163637130000001, 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00222784, 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.978739+0000", 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.978739+0000", 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.978741+0000", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2859999999999998e-06 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979020+0000", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027863699999999998 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979054+0000", 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4329999999999998e-05 2024-09-15T21:00:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979071+0000", 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6821999999999999e-05 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979223+0000", 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015240100000000001 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979244+0000", 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1194000000000001e-05 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.980870+0000", 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016255600000000001 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.980967+0000", 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6609999999999998e-05 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=21, reqid=client.4641.0:113, at_version=33'14, trim_to=0'0, roll_forward_to=33'11))", 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.979110+0000", 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.1159919619999998, 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023253660000000002, 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979110+0000", 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979110+0000", 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979111+0000", 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5300000000000003e-07 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979132+0000", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0883999999999999e-05 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979146+0000", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3709e-05 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.979155+0000", 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3959999999999995e-06 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.980185+0000", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001029435 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.980217+0000", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2444000000000003e-05 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981368+0000", 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011505389999999999 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981436+0000", 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8106000000000001e-05 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=22, reqid=client.4641.0:154, at_version=33'3, trim_to=0'0, roll_forward_to=30'2))", 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.312777+0000", 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7823254670000002, 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001681152, 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.312777+0000", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.312777+0000", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.312778+0000", 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.312978+0000", 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020029999999999999 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.313006+0000", 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7413999999999999e-05 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.313020+0000", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4582e-05 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.313154+0000", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000134363 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.313169+0000", 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4479e-05 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.314363+0000", 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001194318 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.314458+0000", 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4857000000000003e-05 2024-09-15T21:00:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4995511769999998, 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:150, at_version=33'15, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.907611+0000", 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.1874918440000002, 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028667190000000002, 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.907611+0000", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.907611+0000", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.907613+0000", 2024-09-15T21:00:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1330000000000002e-06 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908034+0000", 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00042175999999999997 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908095+0000", 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0393999999999998e-05 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908142+0000", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7058999999999998e-05 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908513+0000", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037055900000000001 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908538+0000", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5060999999999999e-05 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.910378+0000", 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0018400529999999999 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.910477+0000", 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9699999999999998e-05 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:151, at_version=33'16, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.911679+0000", 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.183423098, 2024-09-15T21:00:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0021987389999999999, 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911679+0000", 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911679+0000", 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911681+0000", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0200000000000001e-06 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911879+0000", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019733000000000001 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911906+0000", 2024-09-15T21:00:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.775e-05 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.911922+0000", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5364999999999999e-05 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.912088+0000", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016600999999999999 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.912117+0000", 2024-09-15T21:00:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9017e-05 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.913810+0000", 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001693614 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.913878+0000", 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7632999999999994e-05 2024-09-15T21:00:29.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=39, reqid=client.4641.0:152, at_version=33'17, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:26.013221+0000", 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.0818811670000001, 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001704197, 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.179 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013221+0000", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013221+0000", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013223+0000", 2024-09-15T21:00:29.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013271+0000", 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7895000000000001e-05 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013314+0000", 2024-09-15T21:00:29.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2283000000000002e-05 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013339+0000", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5344e-05 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013509+0000", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017011899999999999 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.013539+0000", 2024-09-15T21:00:29.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0284e-05 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.014821+0000", 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012812329999999999 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.014925+0000", 2024-09-15T21:00:29.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010490899999999999 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=40, reqid=client.4641.0:156, at_version=33'18, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:26.015873+0000", 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.0792291249999999, 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001732248, 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.015873+0000", 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.015873+0000", 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.015875+0000", 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.016112+0000", 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023743700000000001 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.016142+0000", 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9947e-05 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.016158+0000", 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5795e-05 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.016273+0000", 2024-09-15T21:00:29.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011529199999999999 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.016290+0000", 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7195000000000001e-05 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.017542+0000", 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012512420000000001 2024-09-15T21:00:29.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.017606+0000", 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3867000000000004e-05 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.189 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:29.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:29.192 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.166+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:29.192 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.166+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:29.192 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:29.192 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:29.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:29.187+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:29.292 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.286+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:29.295 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:29.295 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:29.320 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:29.320 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:29.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:29.347 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:29.347 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:29.348 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=63, attrs_read=0))", 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.273069+0000", 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.076052923999999994, 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.076074172999999995, 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273069+0000", 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273069+0000", 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273070+0000", 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2300000000000001e-07 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273332+0000", 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026193899999999999 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273341+0000", 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4390000000000006e-06 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273346+0000", 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8130000000000003e-06 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.352 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.273467+0000", 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.075655258000000003, 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.075705663000000006, 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273467+0000", 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273467+0000", 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273467+0000", 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0399999999999996e-07 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273597+0000", 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012935 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.273605+0000", 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3520000000000007e-06 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 36/28 ECSubReadReply(tid=62, attrs_read=0))", 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.168971+0000", 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.18015099400000001, 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.18022343800000001, 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.168971+0000", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.168971+0000", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.168973+0000", 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3659999999999999e-06 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.168986+0000", 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3747000000000001e-05 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.169002+0000", 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5951999999999999e-05 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.169011+0000", 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4319999999999999e-06 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 37/28 ECSubReadReply(tid=62, attrs_read=0))", 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.171505+0000", 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.177617733, 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17771378400000001, 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171505+0000", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171505+0000", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171505+0000", 2024-09-15T21:00:29.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2399999999999999e-07 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171512+0000", 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5490000000000001e-06 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171523+0000", 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268000000000001e-05 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.171530+0000", 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4819999999999997e-06 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=63, attrs_read=0))", 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.172173+0000", 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.17694896399999999, 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17706754199999999, 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.361 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172173+0000", 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172173+0000", 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172173+0000", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172173+0000", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172191+0000", 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7575999999999999e-05 2024-09-15T21:00:29.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272259+0000", 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10006778600000001 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.172315+0000", 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.176807249, 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17694927899999999, 2024-09-15T21:00:29.364 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172315+0000", 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172315+0000", 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172316+0000", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0219999999999999e-06 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172457+0000", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140803 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172465+0000", 2024-09-15T21:00:29.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0460000000000006e-06 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172471+0000", 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5230000000000001e-06 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=63, attrs_read=0))", 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.172453+0000", 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.176668882, 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17683321399999999, 2024-09-15T21:00:29.367 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172453+0000", 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172453+0000", 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172454+0000", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9699999999999996e-07 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172745+0000", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000291174 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172755+0000", 2024-09-15T21:00:29.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.815e-06 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172760+0000", 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3369999999999999e-06 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.370 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.173125+0000", 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.17599695200000001, 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17618325600000001, 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173125+0000", 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173125+0000", 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173126+0000", 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5499999999999998e-07 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173270+0000", 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000144411 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173278+0000", 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4739999999999999e-06 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.173281+0000", 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 37/28 ECSubReadReply(tid=62, attrs_read=0))", 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.269532+0000", 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.079590619000000001, 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.079799162000000007, 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269532+0000", 2024-09-15T21:00:29.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269532+0000", 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269532+0000", 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7099999999999998e-07 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269548+0000", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5778999999999999e-05 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269555+0000", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2479999999999999e-06 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.269558+0000", 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.501e-06 2024-09-15T21:00:29.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.271983+0000", 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.077139111999999996, 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.077369640000000003, 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.271983+0000", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.271983+0000", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.271984+0000", 2024-09-15T21:00:29.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.71e-07 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272079+0000", 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5329999999999997e-05 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272090+0000", 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0711e-05 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272099+0000", 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5969999999999992e-06 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.172464+0000", 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.17665871299999999, 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.176913506, 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172464+0000", 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172464+0000", 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172464+0000", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.172464+0000", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272281+0000", 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099817698999999996 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272287+0000", 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5969999999999999e-06 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 37/28 ECSubReadReply(tid=64, attrs_read=0))", 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.272406+0000", 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.076715979000000004, 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.076992936999999997, 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272406+0000", 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272406+0000", 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272407+0000", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4000000000000001e-07 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272526+0000", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118505 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272538+0000", 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1995e-05 2024-09-15T21:00:29.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.272544+0000", 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0650000000000004e-06 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 12 2024-09-15T21:00:29.386 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:29.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:29.388 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.366+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:29.388 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:29.389 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:29.389 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-15T21:00:29.049028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23151155,"num_objects":51,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":177,"num_read_kb":114,"num_write":235,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":371628,"kb_used_data":47284,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132090452,"statfs":{"total":1159641169920,"available":1159260622848,"internally_reserved":0,"allocated":48418816,"data_stored":46463646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":331855163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,8,12,11,31,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22462464,"num_objects":45,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":99,"num_read_kb":50,"num_write":106,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.025206"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.214951+0000","last_change":"2024-09-15T21:00:20.412757+0000","last_active":"2024-09-15T21:00:26.214951+0000","last_peered":"2024-09-15T21:00:26.214951+0000","last_clean":"2024-09-15T21:00:26.214951+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:00:26.214951+0000","last_undegraded":"2024-09-15T21:00:26.214951+0000","last_fullsized":"2024-09-15T21:00:26.214951+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:36.899698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"33'5","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:00:27.229337+0000","last_change":"2024-09-15T21:00:20.215174+0000","last_active":"2024-09-15T21:00:27.229337+0000","last_peered":"2024-09-15T21:00:27.229337+0000","last_clean":"2024-09-15T21:00:27.229337+0000","last_became_active":"2024-09-15T21:00:20.214352+0000","last_became_peered":"2024-09-15T21:00:20.214352+0000","last_unstale":"2024-09-15T21:00:27.229337+0000","last_undegraded":"2024-09-15T21:00:27.229337+0000","last_fullsized":"2024-09-15T21:00:27.229337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.290377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.d","version":"30'2","reported_seq":31,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.114829+0000","last_change":"2024-09-15T21:00:20.424388+0000","last_active":"2024-09-15T21:00:26.114829+0000","last_peered":"2024-09-15T21:00:26.114829+0000","last_clean":"2024-09-15T21:00:26.114829+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:00:26.114829+0000","last_undegraded":"2024-09-15T21:00:26.114829+0000","last_fullsized":"2024-09-15T21:00:26.114829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:18.986420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245760,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"32'6","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.220566+0000","last_change":"2024-09-15T21:00:20.492369+0000","last_active":"2024-09-15T21:00:26.220566+0000","last_peered":"2024-09-15T21:00:26.220566+0000","last_clean":"2024-09-15T21:00:26.220566+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:00:26.220566+0000","last_undegraded":"2024-09-15T21:00:26.220566+0000","last_fullsized":"2024-09-15T21:00:26.220566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:25.581853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":704512,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"33'9","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.115731+0000","last_change":"2024-09-15T21:00:20.234047+0000","last_active":"2024-09-15T21:00:26.115731+0000","last_peered":"2024-09-15T21:00:26.115731+0000","last_clean":"2024-09-15T21:00:26.115731+0000","last_became_active":"2024-09-15T21:00:20.132390+0000","last_became_peered":"2024-09-15T21:00:20.132390+0000","last_unstale":"2024-09-15T21:00:26.115731+0000","last_undegraded":"2024-09-15T21:00:26.115731+0000","last_fullsized":"2024-09-15T21:00:26.115731+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:36.258603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":9,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'2","reported_seq":31,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.216625+0000","last_change":"2024-09-15T21:00:20.309229+0000","last_active":"2024-09-15T21:00:26.216625+0000","last_peered":"2024-09-15T21:00:26.216625+0000","last_clean":"2024-09-15T21:00:26.216625+0000","last_became_active":"2024-09-15T21:00:20.214883+0000","last_became_peered":"2024-09-15T21:00:20.214883+0000","last_unstale":"2024-09-15T21:00:26.216625+0000","last_undegraded":"2024-09-15T21:00:26.216625+0000","last_fullsized":"2024-09-15T21:00:26.216625+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:43.480601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"34'9","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.116672+0000","last_change":"2024-09-15T21:00:20.309599+0000","last_active":"2024-09-15T21:00:26.116672+0000","last_peered":"2024-09-15T21:00:26.116672+0000","last_clean":"2024-09-15T21:00:26.116672+0000","last_became_active":"2024-09-15T21:00:20.308787+0000","last_became_peered":"2024-09-15T21:00:20.308787+0000","last_unstale":"2024-09-15T21:00:26.116672+0000","last_undegraded":"2024-09-15T21:00:26.116672+0000","last_fullsized":"2024-09-15T21:00:26.116672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:33.544599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2867200,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":9,"num_write_kb":3152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.8","version":"33'3","reported_seq":36,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:00:27.546778+0000","last_change":"2024-09-15T21:00:20.618445+0000","last_active":"2024-09-15T21:00:27.546778+0000","last_peered":"2024-09-15T21:00:27.546778+0000","last_clean":"2024-09-15T21:00:27.546778+0000","last_became_active":"2024-09-15T21:00:20.486849+0000","last_became_peered":"2024-09-15T21:00:20.486849+0000","last_unstale":"2024-09-15T21:00:27.546778+0000","last_undegraded":"2024-09-15T21:00:27.546778+0000","last_fullsized":"2024-09-15T21:00:27.546778+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:43.371797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180224,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":154,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.214647+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:26.214647+0000","last_peered":"2024-09-15T21:00:26.214647+0000","last_clean":"2024-09-15T21:00:26.214647+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:26.214647+0000","last_undegraded":"2024-09-15T21:00:26.214647+0000","last_fullsized":"2024-09-15T21:00:26.214647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":41,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.220550+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:26.220550+0000","last_peered":"2024-09-15T21:00:26.220550+0000","last_clean":"2024-09-15T21:00:26.220550+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:26.220550+0000","last_undegraded":"2024-09-15T21:00:26.220550+0000","last_fullsized":"2024-09-15T21:00:26.220550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"33'16","reported_seq":51,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.212691+0000","last_change":"2024-09-15T21:00:20.513991+0000","last_active":"2024-09-15T21:00:26.212691+0000","last_peered":"2024-09-15T21:00:26.212691+0000","last_clean":"2024-09-15T21:00:26.212691+0000","last_became_active":"2024-09-15T21:00:20.411551+0000","last_became_peered":"2024-09-15T21:00:20.411551+0000","last_unstale":"2024-09-15T21:00:26.212691+0000","last_undegraded":"2024-09-15T21:00:26.212691+0000","last_fullsized":"2024-09-15T21:00:26.212691+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.839842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":5,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":7,"num_write":16,"num_write_kb":1872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.114275+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:26.114275+0000","last_peered":"2024-09-15T21:00:26.114275+0000","last_clean":"2024-09-15T21:00:26.114275+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:26.114275+0000","last_undegraded":"2024-09-15T21:00:26.114275+0000","last_fullsized":"2024-09-15T21:00:26.114275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"32'4","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:24.165571+0000","last_change":"2024-09-15T21:00:20.317554+0000","last_active":"2024-09-15T21:00:24.165571+0000","last_peered":"2024-09-15T21:00:24.165571+0000","last_clean":"2024-09-15T21:00:24.165571+0000","last_became_active":"2024-09-15T21:00:20.315752+0000","last_became_peered":"2024-09-15T21:00:20.315752+0000","last_unstale":"2024-09-15T21:00:24.165571+0000","last_undegraded":"2024-09-15T21:00:24.165571+0000","last_fullsized":"2024-09-15T21:00:24.165571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.586889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:00:27.329078+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:27.329078+0000","last_peered":"2024-09-15T21:00:27.329078+0000","last_clean":"2024-09-15T21:00:27.329078+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:27.329078+0000","last_undegraded":"2024-09-15T21:00:27.329078+0000","last_fullsized":"2024-09-15T21:00:27.329078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"34'10","reported_seq":45,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:00:27.446600+0000","last_change":"2024-09-15T21:00:20.789368+0000","last_active":"2024-09-15T21:00:27.446600+0000","last_peered":"2024-09-15T21:00:27.446600+0000","last_clean":"2024-09-15T21:00:27.446600+0000","last_became_active":"2024-09-15T21:00:20.517607+0000","last_became_peered":"2024-09-15T21:00:20.517607+0000","last_unstale":"2024-09-15T21:00:27.446600+0000","last_undegraded":"2024-09-15T21:00:27.446600+0000","last_fullsized":"2024-09-15T21:00:27.446600+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:48.661169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1982464,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":42,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.220588+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:26.220588+0000","last_peered":"2024-09-15T21:00:26.220588+0000","last_clean":"2024-09-15T21:00:26.220588+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:26.220588+0000","last_undegraded":"2024-09-15T21:00:26.220588+0000","last_fullsized":"2024-09-15T21:00:26.220588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"33'6","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.143157+0000","last_change":"2024-09-15T21:00:20.513108+0000","last_active":"2024-09-15T21:00:26.143157+0000","last_peered":"2024-09-15T21:00:26.143157+0000","last_clean":"2024-09-15T21:00:26.143157+0000","last_became_active":"2024-09-15T21:00:20.410469+0000","last_became_peered":"2024-09-15T21:00:20.410469+0000","last_unstale":"2024-09-15T21:00:26.143157+0000","last_undegraded":"2024-09-15T21:00:26.143157+0000","last_fullsized":"2024-09-15T21:00:26.143157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:41.487181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"33'14","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:25.599196+0000","last_change":"2024-09-15T21:00:20.688686+0000","last_active":"2024-09-15T21:00:25.599196+0000","last_peered":"2024-09-15T21:00:25.599196+0000","last_clean":"2024-09-15T21:00:25.599196+0000","last_became_active":"2024-09-15T21:00:20.585838+0000","last_became_peered":"2024-09-15T21:00:20.585838+0000","last_unstale":"2024-09-15T21:00:25.599196+0000","last_undegraded":"2024-09-15T21:00:25.599196+0000","last_fullsized":"2024-09-15T21:00:25.599196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:41.165054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3522560,"num_objects":7,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":14,"num_read_kb":7,"num_write":14,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:24.160825+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:24.160825+0000","last_peered":"2024-09-15T21:00:24.160825+0000","last_clean":"2024-09-15T21:00:24.160825+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:24.160825+0000","last_undegraded":"2024-09-15T21:00:24.160825+0000","last_fullsized":"2024-09-15T21:00:24.160825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'14","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:25.383704+0000","last_change":"2024-09-15T21:00:20.594178+0000","last_active":"2024-09-15T21:00:25.383704+0000","last_peered":"2024-09-15T21:00:25.383704+0000","last_clean":"2024-09-15T21:00:25.383704+0000","last_became_active":"2024-09-15T21:00:20.492192+0000","last_became_peered":"2024-09-15T21:00:20.492192+0000","last_unstale":"2024-09-15T21:00:25.383704+0000","last_undegraded":"2024-09-15T21:00:25.383704+0000","last_fullsized":"2024-09-15T21:00:25.383704+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:25.415522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2916352,"num_objects":7,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":6,"num_write":14,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.039942+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:26.039942+0000","last_peered":"2024-09-15T21:00:26.039942+0000","last_clean":"2024-09-15T21:00:26.039942+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:26.039942+0000","last_undegraded":"2024-09-15T21:00:26.039942+0000","last_fullsized":"2024-09-15T21:00:26.039942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"33'4","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.115640+0000","last_change":"2024-09-15T21:00:20.616106+0000","last_active":"2024-09-15T21:00:26.115640+0000","last_peered":"2024-09-15T21:00:26.115640+0000","last_clean":"2024-09-15T21:00:26.115640+0000","last_became_active":"2024-09-15T21:00:20.313299+0000","last_became_peered":"2024-09-15T21:00:20.313299+0000","last_unstale":"2024-09-15T21:00:26.115640+0000","last_undegraded":"2024-09-15T21:00:26.115640+0000","last_fullsized":"2024-09-15T21:00:26.115640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:21.536461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:24.160730+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:24.160730+0000","last_peered":"2024-09-15T21:00:24.160730+0000","last_clean":"2024-09-15T21:00:24.160730+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:24.160730+0000","last_undegraded":"2024-09-15T21:00:24.160730+0000","last_fullsized":"2024-09-15T21:00:24.160730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"31'6","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:00:24.165795+0000","last_change":"2024-09-15T21:00:20.519302+0000","last_active":"2024-09-15T21:00:24.165795+0000","last_peered":"2024-09-15T21:00:24.165795+0000","last_clean":"2024-09-15T21:00:24.165795+0000","last_became_active":"2024-09-15T21:00:20.416420+0000","last_became_peered":"2024-09-15T21:00:20.416420+0000","last_unstale":"2024-09-15T21:00:24.165795+0000","last_undegraded":"2024-09-15T21:00:24.165795+0000","last_fullsized":"2024-09-15T21:00:24.165795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:33.861767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851968,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":6,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:00:26.316990+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:26.316990+0000","last_peered":"2024-09-15T21:00:26.316990+0000","last_clean":"2024-09-15T21:00:26.316990+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:26.316990+0000","last_undegraded":"2024-09-15T21:00:26.316990+0000","last_fullsized":"2024-09-15T21:00:26.316990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":22560768,"num_objects":47,"num_object_clones":0,"num_object_copies":376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":101,"num_read_kb":50,"num_write":110,"num_write_kb":22496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":47529984,"data_stored":48144384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378634,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29680,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342160,"statfs":{"total":96636764160,"available":96606371840,"internally_reserved":0,"allocated":2719744,"data_stored":2777264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411338,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":6260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":6410240,"data_stored":6222379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411337,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":5308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":5435392,"data_stored":5251083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411336,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29688,"kb_used_data":2664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342152,"statfs":{"total":96636764160,"available":96606363648,"internally_reserved":0,"allocated":2727936,"data_stored":2547908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411336,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30692,"kb_used_data":3668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341148,"statfs":{"total":96636764160,"available":96605335552,"internally_reserved":0,"allocated":3756032,"data_stored":3572452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444041,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30888,"kb_used_data":3856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340952,"statfs":{"total":96636764160,"available":96605134848,"internally_reserved":0,"allocated":3948544,"data_stored":3768516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444042,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30932,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340908,"statfs":{"total":96636764160,"available":96605089792,"internally_reserved":0,"allocated":3993600,"data_stored":3813572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444041,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":4681728,"data_stored":4493342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,7,3,24,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444040,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30576,"kb_used_data":3544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341264,"statfs":{"total":96636764160,"available":96605454336,"internally_reserved":0,"allocated":3629056,"data_stored":3449028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444042,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30956,"kb_used_data":3928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340884,"statfs":{"total":96636764160,"available":96605065216,"internally_reserved":0,"allocated":4022272,"data_stored":3837963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,4,8,7,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444040,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30744,"kb_used_data":3712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341096,"statfs":{"total":96636764160,"available":96605282304,"internally_reserved":0,"allocated":3801088,"data_stored":3616983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444041,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30248,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341592,"statfs":{"total":96636764160,"available":96605790208,"internally_reserved":0,"allocated":3293184,"data_stored":3113156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2945024,"data_stored":2945024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4444160,"data_stored":4444160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2347008,"data_stored":2592768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3661824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3448832,"data_stored":3448832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4771840,"data_stored":4771840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4513792,"data_stored":4513792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3792896,"data_stored":3792896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":3010560,"data_stored":3010560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096000,"data_stored":4096000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":5074944,"data_stored":5320704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5423104,"data_stored":5545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:29.391 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-15T21:00:29.392 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.366+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:29.392 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.366+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:29.392 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.366+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:29.393 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.370+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:29.393 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.370+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:29.393 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.370+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:00:29.393 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.370+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,4,5 2024-09-15T21:00:29.393 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:29.370+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:29.394 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:29.394 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:29.395 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:29.393+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:29.412 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:29.413 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:29.433 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:29.433 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:29.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.471+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:29.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.471+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:00:29.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.475+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,4,5 2024-09-15T21:00:29.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.475+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6 to read 0,1,4,5 2024-09-15T21:00:29.497 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:29.494+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:29.511 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:29.512 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:29.512 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:29.512 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:29.512 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:29.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:29.614 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:29.610+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:29.677 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:29.673+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:29.677 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:29.673+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:00:29.677 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:29.673+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:00:29.683 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.679+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:29.683 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.679+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:29.684 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.679+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:29.696 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.715615294, 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.697 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:00:29.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:00:29.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:29.702 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.7136040489999997, 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.7128523949999996, 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:00:29.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:00:29.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.7127862110000001, 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:29.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:00:29.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:29.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:128 3.7s0 3:e8b6cabe:::smithi02732947-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.5138662869999999, 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51140068500000002, 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 128 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:29.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178183+0000", 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178189+0000", 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178216+0000", 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6353e-05 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278315+0000", 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100099326 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.365033+0000", 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.086718609000000002 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.365192+0000", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158603 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366018+0000", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00082617400000000005 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367592+0000", 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001574085 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689318+0000", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32172605500000001 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689583+0000", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026455899999999999 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:129 3.7s0 3:e8b6cabe:::smithi02732947-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:29.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.4136809259999996, 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61157217500000005, 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 129 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:29.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278370+0000", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6129999999999998e-06 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278383+0000", 2024-09-15T21:00:29.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3026e-05 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278414+0000", 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0857e-05 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278434+0000", 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9924000000000001e-05 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366928+0000", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088493789000000003 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367150+0000", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000222081 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690379+0000", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32322944100000001 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690606+0000", 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022645599999999999 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.889940+0000", 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19933398799999999 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:29.728 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.2075253860000004, 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:00:29.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.734 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.2060884359999999, 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:00:29.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:29.740 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.2060102910000001, 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:00:29.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.1034951580000003, 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:00:29.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:00:29.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.0234484310000003, 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:29.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:00:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:29.753 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.0090603409999996, 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:29.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:00:29.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.002650386, 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.759 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:00:29.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:00:29.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:00:29.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9151243190000002, 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51692596999999996, 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776926+0000", 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5799999999999999e-06 2024-09-15T21:00:29.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776946+0000", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0315000000000001e-05 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776971+0000", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4706999999999999e-05 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776989+0000", 2024-09-15T21:00:29.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8422000000000001e-05 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293605+0000", 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51661528099999998 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293850+0000", 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245665 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9040026230000002, 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60734763000000003, 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:29.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788048+0000", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.063e-06 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788069+0000", 2024-09-15T21:00:29.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1321000000000001e-05 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788097+0000", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8136999999999998e-05 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888193+0000", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100095434 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394534+0000", 2024-09-15T21:00:29.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50634117999999995 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395394+0000", 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00085949500000000005 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:148 3.bs0 3:d37a3ed4:::smithi02732947-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9021596000000001, 2024-09-15T21:00:29.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60489703100000003, 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 148 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789891+0000", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789902+0000", 2024-09-15T21:00:29.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1362000000000001e-05 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789921+0000", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8868e-05 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789935+0000", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4059e-05 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.793875+0000", 2024-09-15T21:00:29.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039398139999999998 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.794018+0000", 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000143635 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.794823+0000", 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080479900000000003 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795864+0000", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010403669999999999 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394660+0000", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.598796632 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394786+0000", 2024-09-15T21:00:29.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012575800000000001 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9020974129999999, 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.778 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:00:29.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:00:29.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8201400359999997, 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62454644199999998, 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871910+0000", 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.55e-06 2024-09-15T21:00:29.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871958+0000", 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7698000000000003e-05 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871989+0000", 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1061e-05 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.872007+0000", 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7934000000000001e-05 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496399+0000", 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62439274499999997 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496455+0000", 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5454000000000001e-05 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8032725960000002, 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60786671999999997, 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888777+0000", 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4e-07 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888795+0000", 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8403999999999999e-05 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888808+0000", 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2359e-05 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888816+0000", 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3359999999999995e-06 2024-09-15T21:00:29.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496583+0000", 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60776654399999996 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496643+0000", 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0136999999999997e-05 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:160 3.2s0 3:48a96439:::smithi02732947-3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e33)", 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.3990322949999996, 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61085863699999998, 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:29.791 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 160 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293018+0000", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.074e-06 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293027+0000", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3820000000000004e-06 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293049+0000", 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2697999999999999e-05 2024-09-15T21:00:29.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293072+0000", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671e-05 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699705+0000", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406633417 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699975+0000", 2024-09-15T21:00:29.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026989499999999998 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.701228+0000", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012527560000000001 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.702476+0000", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001248033 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903693+0000", 2024-09-15T21:00:29.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20121672600000001 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903875+0000", 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018198500000000001 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:29.796 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:29.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:29.779+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:00:29.798 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:29.799 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:29.791+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:29.898 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:29.897+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:29.932 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:29.932 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:29.932 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:29.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:29.999 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:29.997+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:30.064 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 3 pgp_num to 16 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:85: done (6 left) 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:86: copy_from oid 42 from oid 14 current snap is 3 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:77: finishing write tid 4 to smithi02732947-5 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:77: oid 5 updating version 0 to 20 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:77: oid 5 version 20 is already newer than 19 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:77: oid 5 updating version 20 to 21 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 5 v 21 (ObjNum 56 snap 2 seq_num 56) dirty exists 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:77: left oid 5 (ObjNum 56 snap 2 seq_num 56) 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:83: finishing write tid 1 to smithi02732947-9 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:78: finishing write tid 2 to smithi02732947-16 2024-09-15T21:00:30.074 INFO:tasks.rados.rados.0.smithi027.stdout:82: finishing write tid 1 to smithi02732947-45 2024-09-15T21:00:30.075 INFO:tasks.rados.rados.0.smithi027.stdout:83: finishing write tid 2 to smithi02732947-9 2024-09-15T21:00:30.075 INFO:tasks.rados.rados.0.smithi027.stdout:83: finishing write tid 3 to smithi02732947-9 2024-09-15T21:00:30.075 INFO:tasks.rados.rados.0.smithi027.stdout:78: finishing write tid 3 to smithi02732947-16 2024-09-15T21:00:30.075 INFO:tasks.rados.rados.0.smithi027.stdout:84: expect (ObjNum 30 snap 0 seq_num 30) 2024-09-15T21:00:30.075 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:30.077 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:30.077 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:30.077 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:30.083 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:30.091 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:30.096 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:83: finishing write tid 4 to smithi02732947-9 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:83: oid 9 updating version 0 to 8 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:83: oid 9 updating version 8 to 9 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:83: oid 9 updating version 9 to 10 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 10 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:83: left oid 9 (ObjNum 59 snap 3 seq_num 59) 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:82: finishing write tid 2 to smithi02732947-45 2024-09-15T21:00:30.098 INFO:tasks.rados.rados.0.smithi027.stdout:77: done (6 left) 2024-09-15T21:00:30.099 INFO:tasks.rados.rados.0.smithi027.stdout:83: done (5 left) 2024-09-15T21:00:30.099 INFO:tasks.rados.rados.0.smithi027.stdout:84: done (4 left) 2024-09-15T21:00:30.099 INFO:tasks.rados.rados.0.smithi027.stdout:87: read oid 15 snap -1 2024-09-15T21:00:30.099 INFO:tasks.rados.rados.0.smithi027.stdout:87: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T21:00:30.099 INFO:tasks.rados.rados.0.smithi027.stdout:88: snap_remove snap 2 2024-09-15T21:00:30.100 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:30.097+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:30.201 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:30.194+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:30.203 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:30.203 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:30.204 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.204 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:30.204 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:30.204 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.199+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.229 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.4559209109999998, 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.230 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:30.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:00:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:00:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.233 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.234 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.236 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.237 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.4548195450000003, 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.238 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.239 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.240 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.241 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:00:30.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.244 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.4547790410000001, 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.245 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:00:30.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.4342275320000004, 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.251 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:00:30.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.256 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.365939934, 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:00:30.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.3329206759999996, 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:00:30.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:00:30.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.332880286, 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.270 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.271 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:00:30.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:00:30.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.275 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2646681209999997, 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:00:30.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2627653160000003, 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:00:30.280 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.284 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2607069869999998, 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202146044, 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965234+0000", 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7099999999999999e-06 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965259+0000", 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5732999999999999e-05 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965292+0000", 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2223e-05 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965310+0000", 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8087000000000001e-05 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167355+0000", 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202045846 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167378+0000", 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2444999999999999e-05 2024-09-15T21:00:30.288 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2507832490000004, 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:00:30.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:00:30.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:00:30.294 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.2499735090000001, 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:00:30.295 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:00:30.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.299 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1500418049999999, 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:00:30.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1303657740000004, 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:30.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:00:30.308 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=6, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.8462579100000003, 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19803510199999999, 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.309 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379683+0000", 2024-09-15T21:00:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.917e-06 2024-09-15T21:00:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379693+0000", 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.01e-05 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379722+0000", 2024-09-15T21:00:30.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8623e-05 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379740+0000", 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8677000000000001e-05 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577690+0000", 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19795005800000001 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577716+0000", 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5726999999999999e-05 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.838924059, 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:00:30.314 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:30.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.318 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6415009930000002, 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:00:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:00:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:00:30.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=18, reqid=client.4641.0:143, at_version=33'4, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6371156930000001, 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013446600000001, 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588825+0000", 2024-09-15T21:00:30.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588950+0000", 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124919 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588978+0000", 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8500000000000002e-05 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588993+0000", 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5237e-05 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785673+0000", 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19668001800000001 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785700+0000", 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7112999999999999e-05 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788921+0000", 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032205490000000001 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788958+0000", 2024-09-15T21:00:30.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6619999999999998e-05 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.5582651849999998, 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.330 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 33/28 ECSubReadReply(tid=8, attrs_read=0))", 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.5390663770000002, 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19621026899999999, 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686874+0000", 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7999999999999999e-06 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686892+0000", 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7258e-05 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686916+0000", 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4091999999999999e-05 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686928+0000", 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2242000000000001e-05 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.883066+0000", 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19613826500000001 2024-09-15T21:00:30.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.883083+0000", 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6612e-05 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.340 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:30.342 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.270+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:30.342 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.270+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:30.342 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.274+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:30.343 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.338+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:30.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:30.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.295+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:30.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.295+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:30.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.295+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.399+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.399+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:30.399+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:00:30.442 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:30.441+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:30.482 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=72, last_complete=38'32, committed=1, applied=1))", 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.384779+0000", 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.094952968999999998, 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.094987428999999998, 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384779+0000", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384779+0000", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384780+0000", 2024-09-15T21:00:30.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1230000000000001e-06 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384788+0000", 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384817+0000", 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9196999999999998e-05 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384830+0000", 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.216e-05 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.384851+0000", 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.094880500000000006, 2024-09-15T21:00:30.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.094988988999999996, 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384851+0000", 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384851+0000", 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384852+0000", 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384856+0000", 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1060000000000002e-06 2024-09-15T21:00:30.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384893+0000", 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7809999999999999e-05 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384914+0000", 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0486999999999999e-05 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.489 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=71, last_complete=38'30, committed=1, applied=1))", 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.384984+0000", 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.094747567000000005, 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.094919198999999996, 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384984+0000", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384984+0000", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384985+0000", 2024-09-15T21:00:30.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.569999999999999e-07 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384990+0000", 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385030+0000", 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9972999999999998e-05 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385046+0000", 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5826999999999999e-05 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=72, last_complete=38'32, committed=1, applied=1))", 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.385073+0000", 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.094659008000000003, 2024-09-15T21:00:30.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.094892243000000001, 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385073+0000", 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385073+0000", 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385073+0000", 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6700000000000002e-07 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385080+0000", 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.883e-06 2024-09-15T21:00:30.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385099+0000", 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.859e-05 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385144+0000", 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5602999999999997e-05 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.496 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=71, last_complete=38'30, committed=1, applied=1))", 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.182358+0000", 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.29737395599999999, 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29766937399999999, 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.497 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.182358+0000", 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.182358+0000", 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.182359+0000", 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.55e-06 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.182368+0000", 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9500000000000007e-06 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.182395+0000", 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6906999999999998e-05 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282485+0000", 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100090004 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.500 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=71, last_complete=38'30, committed=1, applied=1))", 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.185167+0000", 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.29456504100000003, 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29492489799999999, 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185167+0000", 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185167+0000", 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185167+0000", 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4300000000000003e-07 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185170+0000", 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8229999999999999e-06 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185184+0000", 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4324e-05 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.185196+0000", 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2109999999999999e-05 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.504 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=72, last_complete=38'32, committed=1, applied=1))", 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.282528+0000", 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19720401900000001, 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19762468899999999, 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282528+0000", 2024-09-15T21:00:30.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282528+0000", 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282529+0000", 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282536+0000", 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0339999999999999e-06 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282555+0000", 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9752999999999999e-05 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282566+0000", 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.076e-05 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s3 38/28 ECSubWrite(tid=35, reqid=osd.1.0:35, at_version=38'25, trim_to=0'0, roll_forward_to=36'23))", 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.282580+0000", 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19715132599999999, 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19763667900000001, 2024-09-15T21:00:30.508 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282580+0000", 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282580+0000", 2024-09-15T21:00:30.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282581+0000", 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3700000000000001e-07 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282614+0000", 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2959000000000001e-05 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282627+0000", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3689999999999999e-05 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282652+0000", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4294000000000002e-05 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385149+0000", 2024-09-15T21:00:30.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102497386 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.385162+0000", 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2709e-05 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.512 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.ds1 38/28 ECSubRead(tid=98, to_read={3:b220672c:::smithi02732947-15:head=0,61440,0}, subchunks={3:b220672c:::smithi02732947-15:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.282851+0000", 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19688055700000001, 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.197451973, 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282851+0000", 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282851+0000", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282852+0000", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.46e-06 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282865+0000", 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2670000000000001e-05 2024-09-15T21:00:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282890+0000", 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5139000000000001e-05 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282905+0000", 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4667e-05 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.518 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.ds1 38/28 ECSubRead(tid=99, to_read={3:b220672c:::smithi02732947-15:head=20480,40960,0}, subchunks={3:b220672c:::smithi02732947-15:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.282939+0000", 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19679218800000001, 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.197447277, 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282939+0000", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282939+0000", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282940+0000", 2024-09-15T21:00:30.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282951+0000", 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.282976+0000", 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5395999999999999e-05 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283003+0000", 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7053999999999999e-05 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.ds1 38/28 ECSubRead(tid=100, to_read={3:b220672c:::smithi02732947-15:head=0,61440,0}, subchunks={3:b220672c:::smithi02732947-15:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.283037+0000", 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.196695022, 2024-09-15T21:00:30.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19742135999999999, 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283037+0000", 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283037+0000", 2024-09-15T21:00:30.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283037+0000", 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2699999999999999e-07 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283047+0000", 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7459999999999995e-06 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283075+0000", 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.792e-05 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383167+0000", 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100092124 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.283883+0000", 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19584850000000001, 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.196635635, 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283883+0000", 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283883+0000", 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283884+0000", 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.158e-06 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283890+0000", 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8420000000000001e-06 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283906+0000", 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5522000000000001e-05 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.283913+0000", 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9349999999999994e-06 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=71, last_complete=38'30, committed=1, applied=1))", 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.284625+0000", 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19510675599999999, 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19595464000000001, 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.529 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.284625+0000", 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.284625+0000", 2024-09-15T21:00:30.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.284626+0000", 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.540000000000001e-07 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.284635+0000", 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5480000000000007e-06 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.284651+0000", 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5515e-05 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.384732+0000", 2024-09-15T21:00:30.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100081687 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.286245+0000", 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.19348663599999999, 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.194396446, 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.533 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286245+0000", 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286245+0000", 2024-09-15T21:00:30.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286246+0000", 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.018e-06 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286251+0000", 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1200000000000001e-06 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286265+0000", 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4076e-05 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.286272+0000", 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8970000000000002e-06 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 38/28 ECSubWriteReply(tid=70, last_complete=38'12, committed=1, applied=1))", 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.379671+0000", 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.100060728, 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10103198100000001, 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379671+0000", 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379671+0000", 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379674+0000", 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7460000000000001e-06 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379689+0000", 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5903999999999999e-05 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379736+0000", 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-05 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379759+0000", 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3329999999999999e-05 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=72, last_complete=38'32, committed=1, applied=1))", 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.380107+0000", 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.099625031000000003, 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100657937, 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.541 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380107+0000", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380107+0000", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380109+0000", 2024-09-15T21:00:30.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5129999999999999e-06 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380121+0000", 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.218e-05 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380163+0000", 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1427000000000002e-05 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380185+0000", 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.23e-05 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.381132+0000", 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.098599853000000001, 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099694405999999999, 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381132+0000", 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381132+0000", 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381133+0000", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1030000000000001e-06 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381140+0000", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7939999999999997e-06 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381164+0000", 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4411999999999999e-05 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.381175+0000", 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0767999999999999e-05 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.383227+0000", 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.096504725, 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097660502999999996, 2024-09-15T21:00:30.547 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383227+0000", 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383227+0000", 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383228+0000", 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.47e-07 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383232+0000", 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2760000000000002e-06 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383249+0000", 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6886000000000001e-05 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383258+0000", 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4709999999999996e-06 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 38/28 ECSubWriteReply(tid=73, last_complete=38'34, committed=1, applied=1))", 2024-09-15T21:00:30.550 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.383373+0000", 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.096358588999999994, 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097576106999999995, 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383373+0000", 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383373+0000", 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383374+0000", 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7400000000000002e-07 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383378+0000", 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.189e-06 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383395+0000", 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.713e-05 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.383424+0000", 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9229000000000001e-05 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:238 3.5s0 3:a50cee1c:::smithi02732947-42:head [copy-from ver 10 in=55b] snapc 4=[4,3] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:30.553 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.405826933, 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40712472999999999, 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 238 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073908+0000", 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1200000000000002e-06 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073922+0000", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4392999999999999e-05 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073954+0000", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1439000000000002e-05 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073974+0000", 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0922999999999999e-05 2024-09-15T21:00:30.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.074945+0000", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097046399999999998 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.075186+0000", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024103199999999999 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.281399+0000", 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20621348 2024-09-15T21:00:30.557 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 20 2024-09-15T21:00:30.558 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:30.559 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:30.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:30.555+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:30.661 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.658+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:30.733 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:30.734 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:30.734 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:30.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:30.762 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:30.761+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:30.863 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:30.861+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:30.873 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:121 3.5s0 3:ac8a7abe:::smithi02732947-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7085565899999997, 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82723362600000006, 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.874 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 121 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161396+0000", 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3630000000000001e-06 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162612+0000", 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001215556 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162641+0000", 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.904e-05 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162654+0000", 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.33e-05 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262983+0000", 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100329088 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.877 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263223+0000", 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023989200000000001 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265887+0000", 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002663978 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.878 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369669+0000", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10378166899999999 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988509+0000", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61884030999999995 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988628+0000", 2024-09-15T21:00:30.879 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011843 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:122 3.5s0 3:ac8a7abe:::smithi02732947-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.707266475, 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82610852099999998, 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.880 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 122 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162685+0000", 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2500000000000001e-06 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162691+0000", 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2900000000000002e-06 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.882 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162703+0000", 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1902999999999999e-05 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162711+0000", 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3000000000000002e-06 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473128+0000", 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31041744100000002 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.883 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473278+0000", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014961900000000001 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474169+0000", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089135400000000002 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587329+0000", 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113159912 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988729+0000", 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40140014600000001 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988793+0000", 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3305999999999993e-05 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.885 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7071941720000003, 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.886 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:00:30.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.888 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.890 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.7071518059999997, 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:00:30.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:00:30.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:00:30.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.706760879, 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:00:30.896 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.897 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.898 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:00:30.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:00:30.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.706721602, 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.902 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:30.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.904 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:00:30.907 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.6674898469999997, 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.908 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:00:30.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:30.913 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.6667674950000002, 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:30.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:00:30.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:00:30.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.666751004, 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.919 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:00:30.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:30.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:00:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:00:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:00:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:00:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.925 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=29, last_complete=33'11, committed=1, applied=1))", 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.678330+0000", 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1916202350000002, 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61275933900000001, 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678330+0000", 2024-09-15T21:00:30.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678330+0000", 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678332+0000", 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5060000000000001e-06 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678344+0000", 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1756999999999999e-05 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678381+0000", 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7333000000000003e-05 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778525+0000", 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100143936 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291068+0000", 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51254263899999997 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291090+0000", 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2167999999999998e-05 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=32, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:30.929 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.687675+0000", 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1822759950000004, 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60316872600000004, 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687675+0000", 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687675+0000", 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687676+0000", 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4900000000000004e-07 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687681+0000", 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9309999999999999e-06 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687694+0000", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3406e-05 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.687701+0000", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9539999999999998e-06 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.290780+0000", 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60307922300000005 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.290843+0000", 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3262999999999994e-05 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=32, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.688211+0000", 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1817396679999996, 2024-09-15T21:00:30.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60279723100000004, 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688211+0000", 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688211+0000", 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688212+0000", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688217+0000", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.78e-06 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688231+0000", 2024-09-15T21:00:30.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.446e-05 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.688239+0000", 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4370000000000001e-06 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.290980+0000", 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60274138700000002 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291008+0000", 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8016000000000001e-05 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1735636439999997, 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:00:30.943 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1671221799999998, 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.944 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:00:30.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:00:30.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:30.949 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1666957780000002, 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:00:30.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:00:30.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=31, last_complete=33'13, committed=1, applied=1))", 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.778637+0000", 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0913136909999999, 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51270318299999995, 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778637+0000", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778637+0000", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778637+0000", 2024-09-15T21:00:30.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5699999999999998e-07 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778652+0000", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4912999999999999e-05 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778666+0000", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.361e-05 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778673+0000", 2024-09-15T21:00:30.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3370000000000002e-06 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291323+0000", 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51264947500000002 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291340+0000", 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7391e-05 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=29, last_complete=33'11, committed=1, applied=1))", 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.778966+0000", 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0909843490000002, 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51308057200000001, 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778966+0000", 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778966+0000", 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778967+0000", 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7599999999999997e-07 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778972+0000", 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2499999999999997e-06 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.778994+0000", 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1557e-05 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779008+0000", 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4446e-05 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291569+0000", 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51256135300000005 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292047+0000", 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00047739000000000001 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=30, last_complete=33'12, committed=1, applied=1))", 2024-09-15T21:00:30.962 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.779029+0000", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0909220299999998, 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51317380199999996, 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779029+0000", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779029+0000", 2024-09-15T21:00:30.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779029+0000", 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779034+0000", 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6260000000000003e-06 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779057+0000", 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3133999999999999e-05 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779069+0000", 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.208e-05 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292149+0000", 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51307936200000004 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292202+0000", 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3869999999999998e-05 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/28 ECSubWriteReply(tid=31, last_complete=33'13, committed=1, applied=1))", 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.779090+0000", 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0908611199999996, 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51323313100000001, 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779090+0000", 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779090+0000", 2024-09-15T21:00:30.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779090+0000", 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8599999999999998e-07 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779106+0000", 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5400000000000002e-05 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779123+0000", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7833999999999999e-05 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.779135+0000", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1878999999999999e-05 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292271+0000", 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51313576299999997 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292323+0000", 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1569e-05 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:238 3.5s0 3:a50cee1c:::smithi02732947-42:head [copy-from ver 10 in=55b] snapc 4=[4,3] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.79604602999999996, 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.711200465, 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:30.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 238 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073908+0000", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1200000000000002e-06 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073922+0000", 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4392999999999999e-05 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073954+0000", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1439000000000002e-05 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073974+0000", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0922999999999999e-05 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.074945+0000", 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097046399999999998 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.075186+0000", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024103199999999999 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.281399+0000", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20621348 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.483210+0000", 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20181083799999999 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.784991+0000", 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301780257 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.785105+0000", 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011451900000000001 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:30.975 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:30.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:30.978 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:30.978+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:31.071 INFO:tasks.rados.rados.0.smithi027.stdout:88: done (5 left) 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:89: copy_from oid 41 from oid 39 current snap is 3 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:80: finishing rollback tid 0 to smithi02732947-27 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:80: finishing rollback tid 1 to smithi02732947-27 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: finishing write tid 3 to smithi02732947-45 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: finishing write tid 4 to smithi02732947-45 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: oid 45 updating version 0 to 12 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: oid 45 version 12 is already newer than 11 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: oid 45 updating version 12 to 13 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 45 v 13 (ObjNum 58 snap 3 seq_num 58) dirty exists 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:82: left oid 45 (ObjNum 58 snap 3 seq_num 58) 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:78: finishing write tid 4 to smithi02732947-16 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:78: oid 16 updating version 0 to 22 2024-09-15T21:00:31.072 INFO:tasks.rados.rados.0.smithi027.stdout:78: oid 16 updating version 22 to 24 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:78: oid 16 version 24 is already newer than 23 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 24 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:78: left oid 16 (ObjNum 57 snap 2 seq_num 57) 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:80: finishing rollback tid 2 to smithi02732947-27 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 27 v 28 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:86: finishing copy_from racing read to smithi02732947-42 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:87: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T21:00:31.073 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:31.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:00:31.079 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:31.077+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:31.079 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 89627 2024-09-15T21:00:31.083 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:31.098 INFO:tasks.rados.rados.0.smithi027.stdout:86: finishing copy_from to smithi02732947-42 2024-09-15T21:00:31.098 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:31.094+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:31.098 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:31.094+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:31.098 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:31.094+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 10 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:78: done (5 left) 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:80: done (4 left) 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:82: done (3 left) 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:86: done (2 left) 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:87: done (1 left) 2024-09-15T21:00:31.099 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 30 current snap is 3 2024-09-15T21:00:31.100 INFO:tasks.rados.rados.0.smithi027.stdout:90: seq_num 60 ranges {491520=638976,1130496=376832} 2024-09-15T21:00:31.106 INFO:tasks.rados.rados.0.smithi027.stdout:90: writing smithi02732947-30 from 491520 to 1130496 tid 1 2024-09-15T21:00:31.109 INFO:tasks.rados.rados.0.smithi027.stdout:90: writing smithi02732947-30 from 1130496 to 1507328 tid 2 2024-09-15T21:00:31.109 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 9 current snap is 3 2024-09-15T21:00:31.109 INFO:tasks.rados.rados.0.smithi027.stdout:91: seq_num 61 ranges {1146880=655360,1802240=376832} 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 39/28 ECSubWrite(tid=37, reqid=client.4641.0:246, at_version=39'26, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.110022+0000", 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0033873110000000001, 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034023629999999998, 2024-09-15T21:00:31.114 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110022+0000", 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110022+0000", 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110022+0000", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7800000000000001e-07 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110418+0000", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000395413 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110428+0000", 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0575e-05 2024-09-15T21:00:31.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.110435+0000", 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:00:31.117 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:31.121 INFO:tasks.rados.rados.0.smithi027.stdout:91: writing smithi02732947-9 from 1146880 to 1802240 tid 1 2024-09-15T21:00:31.124 INFO:tasks.rados.rados.0.smithi027.stdout:91: writing smithi02732947-9 from 1802240 to 2179072 tid 2 2024-09-15T21:00:31.124 INFO:tasks.rados.rados.0.smithi027.stdout:92: rollback oid 21 current snap is 3 2024-09-15T21:00:31.124 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 21 to 1 2024-09-15T21:00:31.124 INFO:tasks.rados.rados.0.smithi027.stdout:89: finishing copy_from racing read to smithi02732947-41 2024-09-15T21:00:31.124 INFO:tasks.rados.rados.0.smithi027.stdout:93: delete oid 17 current snap is 3 2024-09-15T21:00:31.129 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:31.179 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:31.175+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:31.258 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:31.258 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:31.259 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:31.259 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:31.259 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:31.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:31.315 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:31.311+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:31.375 INFO:tasks.rados.rados.0.smithi027.stdout:90: finishing write tid 1 to smithi02732947-30 2024-09-15T21:00:31.380 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:31.375+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 32/28 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-09-15T21:00:31.447 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.4791135459999998, 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.130100361, 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965251+0000", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965252+0000", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0419999999999999e-06 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965258+0000", 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7620000000000001e-06 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965274+0000", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6050999999999999e-05 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065378+0000", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10010451400000001 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094985+0000", 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029606517999999998 2024-09-15T21:00:31.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095351+0000", 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036647399999999999 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 32/28 ECSubWrite(tid=27, reqid=client.4641.0:115, at_version=32'11, trim_to=0'0, roll_forward_to=30'10))", 2024-09-15T21:00:31.451 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.4674139830000001, 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.117700763, 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976951+0000", 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.976952+0000", 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.035e-06 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977149+0000", 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000197061 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977164+0000", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5732000000000001e-05 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.977175+0000", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0995999999999999e-05 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077321+0000", 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10014511399999999 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.077329+0000", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9630000000000004e-06 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094485+0000", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.017155026 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.094651+0000", 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016683600000000001 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.455 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 32/28 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.3789246740000003, 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113605911, 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065440+0000", 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065441+0000", 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1030000000000001e-06 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065449+0000", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1170000000000004e-06 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065471+0000", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1846000000000001e-05 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065483+0000", 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1579999999999999e-05 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.178239+0000", 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11275655800000001 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.179046+0000", 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00080670700000000004 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.459 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1639907940000001, 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20052589700000001, 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.460 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280375+0000", 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.567e-06 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280390+0000", 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4800000000000001e-05 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280411+0000", 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1009000000000002e-05 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280423+0000", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1717e-05 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480839+0000", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20041608799999999 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480900+0000", 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0715999999999999e-05 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.463 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1639144909999999, 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:00:31.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1621839989999998, 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:31.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.471 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1609364209999997, 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:00:31.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:00:31.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:31.475 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.147052317, 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:31.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:00:31.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1469942130000002, 2024-09-15T21:00:31.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.183711915, 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297370+0000", 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297371+0000", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5999999999999999e-07 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297374+0000", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.213e-06 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297382+0000", 2024-09-15T21:00:31.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5549999999999997e-06 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297387+0000", 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9599999999999999e-06 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481022+0000", 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.18363590399999999 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481082+0000", 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9823e-05 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.0632508710000002, 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20016901200000001, 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.483 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381115+0000", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4559999999999999e-06 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381122+0000", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381146+0000", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3731999999999999e-05 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381155+0000", 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0340000000000002e-06 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581244+0000", 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20008901500000001 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581283+0000", 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8884000000000002e-05 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.063170317, 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:31.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:31.490 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.9692919619999998, 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10645436699999999, 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475073+0000", 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475074+0000", 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.164e-06 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475083+0000", 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3999999999999998e-06 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475108+0000", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4803e-05 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475125+0000", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6690000000000001e-05 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581408+0000", 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106283298 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581527+0000", 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011901200000000001 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.9607233690000001, 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:00:31.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.959804471, 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:00:31.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:00:31.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.9572812019999999, 2024-09-15T21:00:31.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:00:31.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.506 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.9549841639999999, 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:31.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:00:31.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:00:31.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:00:31.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:00:31.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs7 33/28 ECSubWrite(tid=44, reqid=client.4641.0:148, at_version=33'9, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.870603+0000", 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.5737614769999997, 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10270853100000001, 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.512 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870603+0000", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870603+0000", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870604+0000", 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870617+0000", 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3148e-05 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870627+0000", 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0502999999999999e-05 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.870636+0000", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9360000000000005e-06 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.971979+0000", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10134273000000001 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.971990+0000", 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0739e-05 2024-09-15T21:00:31.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.973274+0000", 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001284454 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.973312+0000", 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7589999999999998e-05 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.516 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.865387387, 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.8624747070000001, 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:31.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:00:31.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.526 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 38/28 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=38'32, trim_to=0'0, roll_forward_to=37'28))", 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.26596558, 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20162162, 2024-09-15T21:00:31.527 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0100000000000005e-07 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178420+0000", 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0568e-05 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178431+0000", 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0947e-05 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178455+0000", 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3533e-05 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.378594+0000", 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013990700000001 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.378603+0000", 2024-09-15T21:00:31.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2819999999999996e-06 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379987+0000", 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013840250000000001 2024-09-15T21:00:31.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380021+0000", 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3856999999999998e-05 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.535 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.535 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:31.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:31.537 INFO:tasks.rados.rados.0.smithi027.stdout:92: finishing rollback tid 0 to smithi02732947-21 2024-09-15T21:00:31.538 INFO:tasks.rados.rados.0.smithi027.stdout:89: finishing copy_from to smithi02732947-41 2024-09-15T21:00:31.538 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T21:00:31.538 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:31.534+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:31.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:31.629 INFO:tasks.rados.rados.0.smithi027.stdout:92: finishing rollback tid 1 to smithi02732947-21 2024-09-15T21:00:31.629 INFO:tasks.rados.rados.0.smithi027.stdout:92: finishing rollback tid 2 to smithi02732947-21 2024-09-15T21:00:31.629 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 21 v 11 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:89: done (4 left) 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:92: done (3 left) 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:93: done (2 left) 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:94: read oid 44 snap -1 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:94: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 46 current snap is 3 2024-09-15T21:00:31.631 INFO:tasks.rados.rados.0.smithi027.stdout:95: seq_num 63 ranges {770048=688128,1458176=376832} 2024-09-15T21:00:31.637 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:31.638+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:31.646 INFO:tasks.rados.rados.0.smithi027.stdout:95: writing smithi02732947-46 from 770048 to 1458176 tid 1 2024-09-15T21:00:31.650 INFO:tasks.rados.rados.0.smithi027.stdout:95: writing smithi02732947-46 from 1458176 to 1835008 tid 2 2024-09-15T21:00:31.651 INFO:tasks.rados.rados.0.smithi027.stdout:96: rollback oid 13 current snap is 3 2024-09-15T21:00:31.651 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 13 to 1 2024-09-15T21:00:31.651 INFO:tasks.rados.rados.0.smithi027.stdout:97: rollback oid 18 current snap is 3 2024-09-15T21:00:31.651 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 18 to 1 2024-09-15T21:00:31.651 INFO:tasks.rados.rados.0.smithi027.stdout:98: delete oid 19 current snap is 3 2024-09-15T21:00:31.695 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:31.713 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:31.714 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs5 39/28 ECSubWrite(tid=73, reqid=client.4641.0:269, at_version=39'12, trim_to=0'0, roll_forward_to=33'9))", 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.655592+0000", 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.062984837000000002, 2024-09-15T21:00:31.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.063026143000000007, 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.655592+0000", 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.655592+0000", 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:31.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.655594+0000", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0499999999999999e-06 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.655634+0000", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9919999999999997e-05 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.655662+0000", 2024-09-15T21:00:31.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7803000000000001e-05 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:00:31.722 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:31.730 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:31.730 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:31.732 INFO:tasks.rados.rados.0.smithi027.stdout:91: finishing write tid 1 to smithi02732947-9 2024-09-15T21:00:31.732 INFO:tasks.rados.rados.0.smithi027.stdout:91: finishing write tid 2 to smithi02732947-9 2024-09-15T21:00:31.734 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:31.738 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:31.737+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:31.757 INFO:tasks.rados.rados.0.smithi027.stdout:98: done (6 left) 2024-09-15T21:00:31.757 INFO:tasks.rados.rados.0.smithi027.stdout:99: setattr oid 22 current snap is 3 2024-09-15T21:00:31.760 INFO:tasks.rados.rados.0.smithi027.stdout:100: snap_create 2024-09-15T21:00:31.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.771+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:31.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.771+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:31.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.775+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: finishing write tid 3 to smithi02732947-9 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: finishing write tid 4 to smithi02732947-9 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: oid 9 updating version 0 to 15 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: oid 9 updating version 15 to 16 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: oid 9 updating version 16 to 17 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 17 (ObjNum 61 snap 3 seq_num 61) dirty exists 2024-09-15T21:00:31.835 INFO:tasks.rados.rados.0.smithi027.stdout:91: left oid 9 (ObjNum 61 snap 3 seq_num 61) 2024-09-15T21:00:31.840 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:31.837+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:31.852 INFO:tasks.rados.rados.0.smithi027.stdout:90: finishing write tid 2 to smithi02732947-30 2024-09-15T21:00:31.852 INFO:tasks.rados.rados.0.smithi027.stdout:90: finishing write tid 3 to smithi02732947-30 2024-09-15T21:00:31.868 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:31.868 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:31.868 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:31.868 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:31.868 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:31.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:31.884 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:31.885 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:31.912 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:31.912 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:31.934 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.927+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:31.934 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.927+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:31.934 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:31.931+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:31.936 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:31.934+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:31.936 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:31.934+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:00:31.937 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:31.934+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:00:31.938 INFO:tasks.rados.rados.0.smithi027.stdout:90: finishing write tid 4 to smithi02732947-30 2024-09-15T21:00:31.938 INFO:tasks.rados.rados.0.smithi027.stdout:90: oid 30 updating version 0 to 26 2024-09-15T21:00:31.938 INFO:tasks.rados.rados.0.smithi027.stdout:90: oid 30 updating version 26 to 28 2024-09-15T21:00:31.938 INFO:tasks.rados.rados.0.smithi027.stdout:90: oid 30 version 28 is already newer than 27 2024-09-15T21:00:31.938 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 28 (ObjNum 60 snap 3 seq_num 60) dirty exists 2024-09-15T21:00:31.939 INFO:tasks.rados.rados.0.smithi027.stdout:90: left oid 30 (ObjNum 60 snap 3 seq_num 60) 2024-09-15T21:00:31.945 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:31.941+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:31.968 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 19 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:00:32.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.031+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:32.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.031+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:32.042 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:32.042+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:32.058 INFO:tasks.rados.rados.0.smithi027.stdout:97: finishing rollback tid 0 to smithi02732947-18 2024-09-15T21:00:32.058 INFO:tasks.rados.rados.0.smithi027.stdout:97: finishing rollback tid 1 to smithi02732947-18 2024-09-15T21:00:32.058 INFO:tasks.rados.rados.0.smithi027.stdout:95: finishing write tid 1 to smithi02732947-46 2024-09-15T21:00:32.059 INFO:tasks.rados.rados.0.smithi027.stdout:95: finishing write tid 2 to smithi02732947-46 2024-09-15T21:00:32.060 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:32.060 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:32.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 34.597870239000002, 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33373998100000002, 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457904+0000", 2024-09-15T21:00:32.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:00:32.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457918+0000", 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4093999999999999e-05 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457944+0000", 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5986000000000002e-05 2024-09-15T21:00:32.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457965+0000", 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0954e-05 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458117+0000", 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015210500000000001 2024-09-15T21:00:32.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458129+0000", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.149e-05 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539011+0000", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.080882495999999998 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539035+0000", 2024-09-15T21:00:32.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3439999999999999e-05 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788238+0000", 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.24920330900000001 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788633+0000", 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039541700000000001 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.789053+0000", 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00041907000000000001 2024-09-15T21:00:32.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791200+0000", 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021477969999999999 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791447+0000", 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024629799999999999 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791572+0000", 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012488899999999999 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791641+0000", 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9843000000000002e-05 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.072 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.437521373999999, 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.073 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:00:32.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:00:32.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.824227114999999, 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:00:32.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:00:32.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.824042413000001, 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.088 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:00:32.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:00:32.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:00:32.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.618766401999999, 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:00:32.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.618441240999999, 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.101 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:00:32.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.618400384000001, 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:00:32.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.618351638, 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:32.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:00:32.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:00:32.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.618036727, 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.119 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:00:32.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:00:32.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.123 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.124 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.617958660999999, 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.125 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:32.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:00:32.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:00:32.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.923500344000001, 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31281666299999999, 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 71 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132272+0000", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132278+0000", 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7570000000000001e-06 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132291+0000", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2703e-05 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132299+0000", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0069999999999997e-06 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139452+0000", 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071530079999999998 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139562+0000", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000110432 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139743+0000", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018057899999999999 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142717+0000", 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002974661 2024-09-15T21:00:32.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444884+0000", 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216663399999999 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444969+0000", 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4452999999999999e-05 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445088+0000", 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119449 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.923459711, 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31312449999999997, 2024-09-15T21:00:32.137 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 72 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132313+0000", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132422+0000", 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109033 2024-09-15T21:00:32.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132439+0000", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7586000000000001e-05 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132449+0000", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6199999999999994e-06 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140318+0000", 2024-09-15T21:00:32.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078687559999999993 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140430+0000", 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112589 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140615+0000", 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018495600000000001 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142725+0000", 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021100300000000002 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445273+0000", 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30254736500000001 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445366+0000", 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3249000000000001e-05 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445436+0000", 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0406000000000003e-05 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.923304087999998, 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31325695799999997, 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 73 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132469+0000", 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132475+0000", 2024-09-15T21:00:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4999999999999996e-06 2024-09-15T21:00:32.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132493+0000", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7669999999999999e-05 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132507+0000", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3966e-05 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141793+0000", 2024-09-15T21:00:32.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092860819999999993 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141919+0000", 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000126396 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142125+0000", 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020598799999999999 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.143405+0000", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012801570000000001 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445573+0000", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216763699999999 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445651+0000", 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8239999999999996e-05 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445725+0000", 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3329000000000002e-05 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.152 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.923219303, 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31343764099999999, 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 74 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132554+0000", 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.327e-06 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132565+0000", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0967e-05 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132584+0000", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.984e-05 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232738+0000", 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10015367 2024-09-15T21:00:32.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232890+0000", 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015175200000000001 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233152+0000", 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026189099999999999 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233532+0000", 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038017699999999999 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239440+0000", 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005908122 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445839+0000", 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20639856300000001 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445917+0000", 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8055999999999999e-05 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445990+0000", 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3275999999999994e-05 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.158 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.822728791999999, 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31253853100000001, 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 76 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233045+0000", 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:32.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233061+0000", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6453e-05 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233105+0000", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4063000000000001e-05 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233123+0000", 2024-09-15T21:00:32.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7937000000000001e-05 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239508+0000", 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063845940000000004 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239647+0000", 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013887599999999999 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239834+0000", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018760200000000001 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.243229+0000", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033952729999999999 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545355+0000", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30212512200000002 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545446+0000", 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1941999999999995e-05 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545581+0000", 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013493199999999999 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:104 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.164 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 28.743792415000001, 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30628519900000001, 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:32.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311980+0000", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312116+0000", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000136129 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312137+0000", 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1026000000000002e-05 2024-09-15T21:00:32.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312147+0000", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.594e-06 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316046+0000", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003899062 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316173+0000", 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012724000000000001 2024-09-15T21:00:32.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316430+0000", 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025629800000000002 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.317718+0000", 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287949 2024-09-15T21:00:32.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618117+0000", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30039965099999999 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618203+0000", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5415999999999997e-05 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618264+0000", 2024-09-15T21:00:32.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1820000000000001e-05 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.827502956, 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.171 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:00:32.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:00:32.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.827332912999999, 2024-09-15T21:00:32.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:32.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:00:32.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:240 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.958053611, 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30680979800000002, 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 240 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097721+0000", 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2959999999999999e-06 2024-09-15T21:00:32.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097740+0000", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8397e-05 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097776+0000", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6189999999999997e-05 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.197913+0000", 2024-09-15T21:00:32.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10013675700000001 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198064+0000", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015111200000000001 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198219+0000", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000154823 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.404528+0000", 2024-09-15T21:00:32.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20630922300000001 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:242 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.8577525619999999, 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30841185399999999, 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:32.188 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 242 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:32.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198020+0000", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000002e-07 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198024+0000", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4100000000000001e-06 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198035+0000", 2024-09-15T21:00:32.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0507e-05 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198058+0000", 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3606000000000001e-05 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.201317+0000", 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003258862 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.201451+0000", 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013384200000000001 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.506431+0000", 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30497976700000001 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:32.192 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:32.193 INFO:tasks.rados.rados.0.smithi027.stdout:97: finishing rollback tid 2 to smithi02732947-18 2024-09-15T21:00:32.194 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 13 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T21:00:32.194 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:00:32.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:32.196 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.131+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:90: done (7 left) 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:91: done (6 left) 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:97: done (5 left) 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:99: done (4 left) 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:100: done (3 left) 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:101: copy_from oid 17 from oid 50 current snap is 4 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 38 current snap is 4 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:102: seq_num 65 ranges {622592=720896,1343488=376832} 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:102: writing smithi02732947-38 from 622592 to 1343488 tid 1 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:102: writing smithi02732947-38 from 1343488 to 1720320 tid 2 2024-09-15T21:00:32.197 INFO:tasks.rados.rados.0.smithi027.stdout:103: snap_remove snap 3 2024-09-15T21:00:32.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:32.198 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:32.199 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:32.199 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:32.158+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:32.200 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:32.162+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:00:32.200 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:32.194+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:32.225 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:32.236 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.231+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:32.265 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:32.262+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:32.266 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:32.295 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:32.291+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:32.301 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:32.331 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:32.331 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:32.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:32.334 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:32.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:32.395 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:32.393+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:32.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.435+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:32.469 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:32.469 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:32.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:32.526 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:32.525+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:32.547 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.543+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:32.547 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.543+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:32.547 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.543+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:00:32.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.547+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:32.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.547+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:32.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:32.547+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:00:32.597 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:32.597+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:32.648 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:32.648 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:32.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:32.705 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:32.705+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:32.799 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:32.797+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:32.827 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:32.827 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:32.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:32.900 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:32.898+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:33.002 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:33.001+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:33.010 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.010 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:33.094 INFO:tasks.rados.rados.0.smithi027.stdout:103: done (5 left) 2024-09-15T21:00:33.094 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 18 current snap is 4 2024-09-15T21:00:33.094 INFO:tasks.rados.rados.0.smithi027.stdout:104: seq_num 66 ranges {294912=737280,1032192=376832} 2024-09-15T21:00:33.102 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:33.099+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:33.110 INFO:tasks.rados.rados.0.smithi027.stdout:104: writing smithi02732947-18 from 294912 to 1032192 tid 1 2024-09-15T21:00:33.118 INFO:tasks.rados.rados.0.smithi027.stdout:104: writing smithi02732947-18 from 1032192 to 1409024 tid 2 2024-09-15T21:00:33.118 INFO:tasks.rados.rados.0.smithi027.stdout:105: rollback oid 30 current snap is 4 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 30 to 1 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:95: finishing write tid 3 to smithi02732947-46 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:106: read oid 22 snap -1 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:106: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 5 current snap is 4 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:96: finishing rollback tid 0 to smithi02732947-13 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:95: finishing write tid 4 to smithi02732947-46 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:95: oid 46 updating version 0 to 25 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:95: oid 46 updating version 25 to 27 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:95: oid 46 version 27 is already newer than 26 2024-09-15T21:00:33.119 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 46 v 27 (ObjNum 63 snap 3 seq_num 63) dirty exists 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:95: left oid 46 (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:96: finishing rollback tid 1 to smithi02732947-13 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:96: finishing rollback tid 2 to smithi02732947-13 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 32 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:94: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:33.120 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:33.140 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 98588 2024-09-15T21:00:33.162 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:33.158+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:33.162 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:33.162+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:00:33.162 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:33.162+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4 to read 0,1,4,5 2024-09-15T21:00:33.162 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:33.162+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,7 to read 0,1,4,5 2024-09-15T21:00:33.163 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 261411 2024-09-15T21:00:33.175 INFO:tasks.rados.rados.0.smithi027.stdout:101: finishing copy_from racing read to smithi02732947-17 2024-09-15T21:00:33.175 INFO:tasks.rados.rados.0.smithi027.stdout:102: finishing write tid 1 to smithi02732947-38 2024-09-15T21:00:33.175 INFO:tasks.rados.rados.0.smithi027.stdout:107: seq_num 67 ranges {1032192=753664,1785856=376832} 2024-09-15T21:00:33.192 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.193 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.199 INFO:tasks.rados.rados.0.smithi027.stdout:107: writing smithi02732947-5 from 1032192 to 1785856 tid 1 2024-09-15T21:00:33.203 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.199+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:33.205 INFO:tasks.rados.rados.0.smithi027.stdout:107: writing smithi02732947-5 from 1785856 to 2162688 tid 2 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:94: done (8 left) 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:95: done (7 left) 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:96: done (6 left) 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:108: rollback oid 49 current snap is 4 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 49 to 1 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: finishing write tid 2 to smithi02732947-38 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: finishing write tid 3 to smithi02732947-38 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: finishing write tid 4 to smithi02732947-38 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: oid 38 updating version 0 to 36 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: oid 38 version 36 is already newer than 34 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: oid 38 version 36 is already newer than 35 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 36 (ObjNum 65 snap 4 seq_num 65) dirty exists 2024-09-15T21:00:33.206 INFO:tasks.rados.rados.0.smithi027.stdout:102: left oid 38 (ObjNum 65 snap 4 seq_num 65) 2024-09-15T21:00:33.207 INFO:tasks.rados.rados.0.smithi027.stdout:102: done (6 left) 2024-09-15T21:00:33.207 INFO:tasks.rados.rados.0.smithi027.stdout:109: read oid 15 snap 1 2024-09-15T21:00:33.207 INFO:tasks.rados.rados.0.smithi027.stdout:109: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T21:00:33.207 INFO:tasks.rados.rados.0.smithi027.stdout:110: snap_create 2024-09-15T21:00:33.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:33.221 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.217+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:33.222 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.221+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:00:33.222 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.221+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,4,5 2024-09-15T21:00:33.225 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.221+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:33.225 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.221+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:33.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:33.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:33.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:33.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6 to read 0,1,4,5 2024-09-15T21:00:33.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,6 to read 0,1,4,5 2024-09-15T21:00:33.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.227+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:33.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.231+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,6 to read 0,1,4,5 2024-09-15T21:00:33.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.231+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:33.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.231+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,6 to read 0,1,4,5 2024-09-15T21:00:33.257 INFO:tasks.rados.rados.0.smithi027.stdout:101: finishing copy_from to smithi02732947-17 2024-09-15T21:00:33.257 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 17 v 17 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T21:00:33.305 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.299+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:33.328 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.325+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:33.328 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.325+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:33.328 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.325+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,7 to read 0,1,4,5 2024-09-15T21:00:33.329 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.325+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:00:33.334 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.334 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.331+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,4,5 2024-09-15T21:00:33.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.331+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4 to read 0,1,4,5 2024-09-15T21:00:33.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.331+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,6 to read 0,1,4,5 2024-09-15T21:00:33.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:33.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.399+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:33.507 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.506+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:33.517 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.517 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.530 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.529+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:33.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:33.533 INFO:tasks.rados.rados.0.smithi027.stdout:106: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T21:00:33.533 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:33.563 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 558474 2024-09-15T21:00:33.569 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:33.593 INFO:tasks.rados.rados.0.smithi027.stdout:109: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T21:00:33.593 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:33.598 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 46725 2024-09-15T21:00:33.601 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:33.608 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:33.605+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:33.628 INFO:tasks.rados.rados.0.smithi027.stdout:105: finishing rollback tid 0 to smithi02732947-30 2024-09-15T21:00:33.628 INFO:tasks.rados.rados.0.smithi027.stdout:105: finishing rollback tid 1 to smithi02732947-30 2024-09-15T21:00:33.628 INFO:tasks.rados.rados.0.smithi027.stdout:105: finishing rollback tid 2 to smithi02732947-30 2024-09-15T21:00:33.628 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 39 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T21:00:33.654 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.654 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:33.710 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:33.709+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:33.803+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:33.824 INFO:tasks.rados.rados.0.smithi027.stdout:107: finishing write tid 1 to smithi02732947-5 2024-09-15T21:00:33.824 INFO:tasks.rados.rados.0.smithi027.stdout:104: finishing write tid 1 to smithi02732947-18 2024-09-15T21:00:33.825 INFO:tasks.rados.rados.0.smithi027.stdout:107: finishing write tid 2 to smithi02732947-5 2024-09-15T21:00:33.833 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.834 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.841 INFO:tasks.rados.rados.0.smithi027.stdout:104: finishing write tid 2 to smithi02732947-18 2024-09-15T21:00:33.841 INFO:tasks.rados.rados.0.smithi027.stdout:104: finishing write tid 3 to smithi02732947-18 2024-09-15T21:00:33.842 INFO:tasks.rados.rados.0.smithi027.stdout:108: finishing rollback tid 0 to smithi02732947-49 2024-09-15T21:00:33.844 INFO:tasks.rados.rados.0.smithi027.stdout:108: finishing rollback tid 1 to smithi02732947-49 2024-09-15T21:00:33.844 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.841+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:33.844 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.841+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:33.844 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.841+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:33.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:33.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:33.907+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:33.924 INFO:tasks.rados.rados.0.smithi027.stdout:107: finishing write tid 3 to smithi02732947-5 2024-09-15T21:00:33.926 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.925+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:00:33.927 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.925+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:33.943 INFO:tasks.rados.rados.0.smithi027.stdout:108: finishing rollback tid 2 to smithi02732947-49 2024-09-15T21:00:33.943 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 49 v 20 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T21:00:33.944 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:33.941+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:104: finishing write tid 4 to smithi02732947-18 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:104: oid 18 updating version 0 to 14 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:104: oid 18 updating version 14 to 15 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:104: oid 18 updating version 15 to 16 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 16 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-15T21:00:33.945 INFO:tasks.rados.rados.0.smithi027.stdout:104: left oid 18 (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:00:33.971 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:33.971 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:33.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:34.013 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:34.009+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:34.027 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:34.025+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:00:34.027 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:34.025+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,5 to read 0,1,4,5 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:101: done (7 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:104: done (6 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:105: done (5 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:106: done (4 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:108: done (3 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:109: done (2 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:110: done (1 left) 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 41 current snap is 5 2024-09-15T21:00:34.102 INFO:tasks.rados.rados.0.smithi027.stdout:111: seq_num 68 ranges {638976=770048,1409024=376832} 2024-09-15T21:00:34.109 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:34.109 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:34.107+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:34.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:34.125 INFO:tasks.rados.rados.0.smithi027.stdout:111: writing smithi02732947-41 from 638976 to 1409024 tid 1 2024-09-15T21:00:34.131 INFO:tasks.rados.rados.0.smithi027.stdout:111: writing smithi02732947-41 from 1409024 to 1785856 tid 2 2024-09-15T21:00:34.131 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 13 current snap is 5 2024-09-15T21:00:34.131 INFO:tasks.rados.rados.0.smithi027.stdout:112: seq_num 69 ranges {212992=786432,999424=376832} 2024-09-15T21:00:34.149 INFO:tasks.rados.rados.0.smithi027.stdout:112: writing smithi02732947-13 from 212992 to 999424 tid 1 2024-09-15T21:00:34.155 INFO:tasks.rados.rados.0.smithi027.stdout:112: writing smithi02732947-13 from 999424 to 1376256 tid 2 2024-09-15T21:00:34.155 INFO:tasks.rados.rados.0.smithi027.stdout:113: snap_create 2024-09-15T21:00:34.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:00:34.215 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:34.211+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:34.227 INFO:tasks.rados.rados.0.smithi027.stdout:107: finishing write tid 4 to smithi02732947-5 2024-09-15T21:00:34.227 INFO:tasks.rados.rados.0.smithi027.stdout:107: oid 5 updating version 0 to 30 2024-09-15T21:00:34.227 INFO:tasks.rados.rados.0.smithi027.stdout:107: oid 5 version 30 is already newer than 28 2024-09-15T21:00:34.227 INFO:tasks.rados.rados.0.smithi027.stdout:107: oid 5 version 30 is already newer than 29 2024-09-15T21:00:34.227 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 5 v 30 (ObjNum 67 snap 4 seq_num 67) dirty exists 2024-09-15T21:00:34.228 INFO:tasks.rados.rados.0.smithi027.stdout:107: left oid 5 (ObjNum 67 snap 4 seq_num 67) 2024-09-15T21:00:34.260 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:34.261 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:34.277 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:34.315 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:34.313+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:34.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:34.416 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:34.413+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:34.437 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:34.438 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:34.441 INFO:tasks.rados.rados.0.smithi027.stdout:111: finishing write tid 1 to smithi02732947-41 2024-09-15T21:00:34.441 INFO:tasks.rados.rados.0.smithi027.stdout:111: finishing write tid 2 to smithi02732947-41 2024-09-15T21:00:34.441 INFO:tasks.rados.rados.0.smithi027.stdout:111: finishing write tid 3 to smithi02732947-41 2024-09-15T21:00:34.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:34.517 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:34.513+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:34.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:34.531 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:34.545 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:34.539+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:34.545 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:34.539+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:34.550 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:34.551 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:34.568 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:34.569 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:34.596 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.330628934, 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.598 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:00:34.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.601 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.604 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.330330795, 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:00:34.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:00:34.607 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.608 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.609 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.330306501000001, 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:00:34.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:00:34.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:00:34.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471527+0000", 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.123687124, 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.295011148, 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471527+0000", 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471527+0000", 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471527+0000", 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471527+0000", 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471577+0000", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9885000000000001e-05 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471598+0000", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1804e-05 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.766398+0000", 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29479982700000001 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.766538+0000", 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013963199999999999 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.123244038999999, 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:34.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:00:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.472023+0000", 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.123190603999999, 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29479808800000001, 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472023+0000", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472023+0000", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472025+0000", 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.353e-06 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472069+0000", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4431999999999997e-05 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472093+0000", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4230999999999999e-05 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.572185+0000", 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100091927 2024-09-15T21:00:34.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.766618+0000", 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.194432577 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.766821+0000", 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000203568 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.es4 32/28 ECSubWrite(tid=13, reqid=client.4641.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.047865825000001, 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32195022499999998, 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.630 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547350+0000", 2024-09-15T21:00:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547377+0000", 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6630000000000001e-05 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547411+0000", 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3708999999999999e-05 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547429+0000", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867891+0000", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32046238700000002 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867902+0000", 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1385e-05 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869267+0000", 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013649020000000001 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869298+0000", 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0997000000000002e-05 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.es4 32/28 ECSubRead(tid=14, to_read={3:770058af:::smithi02732947-19:head=0,4096,0}, subchunks={3:770058af:::smithi02732947-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.045473673, 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31956152100000002, 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549742+0000", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.516e-06 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549757+0000", 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5248999999999999e-05 2024-09-15T21:00:34.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549789+0000", 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.205e-05 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549807+0000", 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7870999999999999e-05 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868926+0000", 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31911888799999999 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869302+0000", 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037594700000000001 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-15T21:00:34.638 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.036764212, 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30910945000000001, 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:34.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558451+0000", 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4130000000000001e-06 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558472+0000", 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1038000000000001e-05 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558497+0000", 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4620000000000001e-05 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558512+0000", 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4902000000000001e-05 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867529+0000", 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30901767499999999 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867559+0000", 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9802e-05 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.036323539, 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30970041700000001, 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.642 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558892+0000", 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5e-06 2024-09-15T21:00:34.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558915+0000", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3527e-05 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558944+0000", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8827e-05 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558962+0000", 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7320999999999999e-05 2024-09-15T21:00:34.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868104+0000", 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30914279099999997 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868591+0000", 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048645099999999999 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.035718300999999, 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30937452599999998, 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559501+0000", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.339e-06 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559517+0000", 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6084999999999999e-05 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559537+0000", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9911e-05 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559552+0000", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5424999999999999e-05 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868839+0000", 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30928677999999998 2024-09-15T21:00:34.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868870+0000", 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0985999999999998e-05 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.649 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.565133+0000", 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.030080429, 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30293384800000001, 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565133+0000", 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565133+0000", 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565133+0000", 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565133+0000", 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565169+0000", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5373000000000002e-05 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.565185+0000", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6109000000000001e-05 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867962+0000", 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30277706999999998 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868067+0000", 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000105296 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.657661+0000", 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.937552509, 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21027011200000001, 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657661+0000", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657661+0000", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657663+0000", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2559999999999999e-06 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657691+0000", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8232999999999999e-05 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657715+0000", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4134999999999998e-05 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657733+0000", 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7501000000000001e-05 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867903+0000", 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21017075299999999 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867932+0000", 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8234000000000001e-05 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.657761+0000", 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.937453074, 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.211208804, 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.657 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657761+0000", 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657761+0000", 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657762+0000", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8700000000000004e-07 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657784+0000", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2320999999999999e-05 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657802+0000", 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8198000000000001e-05 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.657817+0000", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4562000000000001e-05 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868940+0000", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.211123229 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868970+0000", 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9607e-05 2024-09-15T21:00:34.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:162 3.6s0 3:6097bd07:::smithi02732947-11:head [read 163964~16260,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.904666+0000", 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.690548282, 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21308671600000001, 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:34.661 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 162 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904666+0000", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904666+0000", 2024-09-15T21:00:34.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904668+0000", 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3539999999999998e-06 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904678+0000", 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0375e-05 2024-09-15T21:00:34.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904701+0000", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2311e-05 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904723+0000", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2444000000000001e-05 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:00:34.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001450069 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.906421+0000", 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024756799999999999 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.117752+0000", 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21133159500000001 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:163 3.6s0 3:6097bd07:::smithi02732947-11:head [read 51284~128940,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.904750+0000", 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.6904643640000003, 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30811013799999998, 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 163 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.667 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904750+0000", 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904750+0000", 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904751+0000", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904762+0000", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1276e-05 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904782+0000", 2024-09-15T21:00:34.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9491e-05 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.904797+0000", 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4849e-05 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908096+0000", 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003299691 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.908306+0000", 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209932 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.212860+0000", 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30455342899999999 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 33/28 ECSubReadReply(tid=21, attrs_read=0))", 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.909477+0000", 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.6857370580000008, 2024-09-15T21:00:34.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30251122699999999, 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909477+0000", 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909477+0000", 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909477+0000", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909477+0000", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909530+0000", 2024-09-15T21:00:34.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3551000000000003e-05 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.909544+0000", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3125999999999999e-05 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.211953+0000", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30240950500000002 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:26.211988+0000", 2024-09-15T21:00:34.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5045000000000001e-05 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.4700079960000001, 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.675 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:00:34.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.680 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.369842937, 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:00:34.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:00:34.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.3697982670000002, 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:00:34.686 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:00:34.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:00:34.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:00:34.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.692 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.692 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:34.693 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:34.639+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:00:34.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:34.693 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:34.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:34.694 INFO:tasks.rados.rados.0.smithi027.stdout:111: finishing write tid 4 to smithi02732947-41 2024-09-15T21:00:34.695 INFO:tasks.rados.rados.0.smithi027.stdout:111: oid 41 updating version 0 to 45 2024-09-15T21:00:34.695 INFO:tasks.rados.rados.0.smithi027.stdout:111: oid 41 version 45 is already newer than 44 2024-09-15T21:00:34.695 INFO:tasks.rados.rados.0.smithi027.stdout:111: oid 41 version 45 is already newer than 43 2024-09-15T21:00:34.695 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 45 (ObjNum 68 snap 5 seq_num 68) dirty exists 2024-09-15T21:00:34.695 INFO:tasks.rados.rados.0.smithi027.stdout:111: left oid 41 (ObjNum 68 snap 5 seq_num 68) 2024-09-15T21:00:34.695 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:34.695 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:34.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:34.696 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:34.696 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:34.687+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:34.711 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:34.732 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:34.745 INFO:tasks.rados.rados.0.smithi027.stdout:112: finishing write tid 1 to smithi02732947-13 2024-09-15T21:00:34.745 INFO:tasks.rados.rados.0.smithi027.stdout:112: finishing write tid 2 to smithi02732947-13 2024-09-15T21:00:34.745 INFO:tasks.rados.rados.0.smithi027.stdout:112: finishing write tid 3 to smithi02732947-13 2024-09-15T21:00:34.753 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:34.754 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:34.774 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:34.774 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:34.794 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:34.793+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:34.794 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:34.832 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:34.832 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:34.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.833 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.833 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.746692+0000", 2024-09-15T21:00:34.833 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.085312684, 2024-09-15T21:00:34.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.085333949000000006, 2024-09-15T21:00:34.834 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:34.834 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746692+0000", 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746692+0000", 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746693+0000", 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2200000000000002e-07 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746697+0000", 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.459e-06 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746709+0000", 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1447000000000001e-05 2024-09-15T21:00:34.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746715+0000", 2024-09-15T21:00:34.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3690000000000003e-06 2024-09-15T21:00:34.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.840 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.746715+0000", 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.085289497000000006, 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.085338555999999996, 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:34.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746715+0000", 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746715+0000", 2024-09-15T21:00:34.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746716+0000", 2024-09-15T21:00:34.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:00:34.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746721+0000", 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9069999999999998e-06 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.746737+0000", 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6130999999999999e-05 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.747666+0000", 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.084338266999999995, 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084407742999999993, 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747666+0000", 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747666+0000", 2024-09-15T21:00:34.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747667+0000", 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747670+0000", 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.932e-06 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747679+0000", 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8750000000000006e-06 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747684+0000", 2024-09-15T21:00:34.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8049999999999997e-06 2024-09-15T21:00:34.850 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.850 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.747713+0000", 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.084291809999999995, 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084383725000000007, 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747713+0000", 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747713+0000", 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747713+0000", 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747715+0000", 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8700000000000001e-06 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747721+0000", 2024-09-15T21:00:34.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2609999999999999e-06 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747725+0000", 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9519999999999996e-06 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.747769+0000", 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.084235619999999997, 2024-09-15T21:00:34.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084387303999999996, 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747769+0000", 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747769+0000", 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747770+0000", 2024-09-15T21:00:34.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1399999999999996e-07 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747774+0000", 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7629999999999998e-06 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747784+0000", 2024-09-15T21:00:34.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0368e-05 2024-09-15T21:00:34.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747789+0000", 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.809e-06 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 42/28 ECSubWriteReply(tid=59, last_complete=42'45, committed=1, applied=1))", 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.747855+0000", 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.084150055000000001, 2024-09-15T21:00:34.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084365334, 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747855+0000", 2024-09-15T21:00:34.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747855+0000", 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747855+0000", 2024-09-15T21:00:34.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747859+0000", 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4709999999999999e-06 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747869+0000", 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0025e-05 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:34.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.747873+0000", 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 6 2024-09-15T21:00:34.866 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:34.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:34.894 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:34.893+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:34.996 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:34.993+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:35.037 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:35.037 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:35.037 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:35.047 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:35.045+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:35.047 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:35.045+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:35.047 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:35.045+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:35.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:35.098 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:35.097+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:35.098 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:35.098 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:35.098 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:35.099 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:00:35.102 INFO:tasks.rados.rados.0.smithi027.stdout:107: done (3 left) 2024-09-15T21:00:35.102 INFO:tasks.rados.rados.0.smithi027.stdout:111: done (2 left) 2024-09-15T21:00:35.102 INFO:tasks.rados.rados.0.smithi027.stdout:113: done (1 left) 2024-09-15T21:00:35.102 INFO:tasks.rados.rados.0.smithi027.stdout:114: snap_create 2024-09-15T21:00:35.148 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:35.145+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:35.148 INFO:tasks.rados.rados.0.smithi027.stdout:112: finishing write tid 4 to smithi02732947-13 2024-09-15T21:00:35.148 INFO:tasks.rados.rados.0.smithi027.stdout:112: oid 13 updating version 0 to 43 2024-09-15T21:00:35.148 INFO:tasks.rados.rados.0.smithi027.stdout:112: oid 13 updating version 43 to 45 2024-09-15T21:00:35.148 INFO:tasks.rados.rados.0.smithi027.stdout:112: oid 13 version 45 is already newer than 44 2024-09-15T21:00:35.148 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 45 (ObjNum 69 snap 5 seq_num 69) dirty exists 2024-09-15T21:00:35.149 INFO:tasks.rados.rados.0.smithi027.stdout:112: left oid 13 (ObjNum 69 snap 5 seq_num 69) 2024-09-15T21:00:35.187 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:35.187 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.429570407, 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.191 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.429523349, 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:00:35.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:00:35.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:00:35.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.429495549, 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:00:35.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.429470155000001, 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:00:35.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:00:35.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.124090933, 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.205 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:35.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:00:35.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:00:35.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.209 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.123529973, 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.211 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:00:35.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.02337159, 2024-09-15T21:00:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:00:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:00:35.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:00:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.023275119999999, 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.227 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:00:35.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:35.233 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.02231769, 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:35.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:35.239 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.022248184, 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:00:35.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:00:35.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.922480479000001, 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:00:35.245 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:00:35.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.818808305999999, 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.805548535, 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:00:35.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:00:35.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:35.257 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.803608108000001, 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:35.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.261 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.613954481, 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:00:35.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:00:35.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.204944052, 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.266 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:00:35.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:00:35.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:00:35.270 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.197422644, 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:00:35.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:00:35.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.004543033999999, 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:00:35.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.003539955999999, 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:00:35.282 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.33898715099999999, 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:35.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.286 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:35.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:35.288 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:35.285+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:35.388 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:35.385+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:35.398 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:35.416 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-09-15T21:00:35.383192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30179891,"num_objects":62,"num_object_clones":11,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":2,"num_read":230,"num_read_kb":11033,"num_write":308,"num_write_kb":32440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":393332,"kb_used_data":64816,"kb_used_omap":18,"kb_used_meta":328237,"kb_avail":1132068748,"statfs":{"total":1159641169920,"available":1159238397952,"internally_reserved":0,"allocated":66371584,"data_stored":71076557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":336115003},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,4,11,8,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6651904,"num_objects":10,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":39,"num_read_kb":8572,"num_write":48,"num_write_kb":7056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001183"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.205128+0000","last_change":"2024-09-15T21:00:31.206087+0000","last_active":"2024-09-15T21:00:31.205128+0000","last_peered":"2024-09-15T21:00:31.205128+0000","last_clean":"2024-09-15T21:00:31.205128+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:00:31.205128+0000","last_undegraded":"2024-09-15T21:00:31.205128+0000","last_fullsized":"2024-09-15T21:00:31.205128+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:35.471753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001789442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.e","version":"39'7","reported_seq":62,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T21:00:32.507725+0000","last_change":"2024-09-15T21:00:31.309803+0000","last_active":"2024-09-15T21:00:32.507725+0000","last_peered":"2024-09-15T21:00:32.507725+0000","last_clean":"2024-09-15T21:00:32.507725+0000","last_became_active":"2024-09-15T21:00:20.214352+0000","last_became_peered":"2024-09-15T21:00:20.214352+0000","last_unstale":"2024-09-15T21:00:32.507725+0000","last_undegraded":"2024-09-15T21:00:32.507725+0000","last_fullsized":"2024-09-15T21:00:32.507725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:29.949780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029707269999999998,"stat_sum":{"num_bytes":1114112,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":4,"num_read_kb":2,"num_write":6,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.d","version":"34'3","reported_seq":58,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.104170+0000","last_change":"2024-09-15T21:00:31.104474+0000","last_active":"2024-09-15T21:00:31.104170+0000","last_peered":"2024-09-15T21:00:31.104170+0000","last_clean":"2024-09-15T21:00:31.104170+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:00:31.104170+0000","last_undegraded":"2024-09-15T21:00:31.104170+0000","last_fullsized":"2024-09-15T21:00:31.104170+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:19.383599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090390499999999999,"stat_sum":{"num_bytes":245760,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":679,"num_write":22,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.c","version":"38'12","reported_seq":65,"reported_epoch":39,"state":"active+clean+snaptrim","last_fresh":"2024-09-15T21:00:31.070376+0000","last_change":"2024-09-15T21:00:31.070421+0000","last_active":"2024-09-15T21:00:31.070376+0000","last_peered":"2024-09-15T21:00:31.070376+0000","last_clean":"2024-09-15T21:00:31.070376+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:00:31.070376+0000","last_undegraded":"2024-09-15T21:00:31.070376+0000","last_fullsized":"2024-09-15T21:00:31.070376+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:44.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50734382499999997,"stat_sum":{"num_bytes":1703936,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":9,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"33'9","reported_seq":67,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.094222+0000","last_change":"2024-09-15T21:00:31.094713+0000","last_active":"2024-09-15T21:00:31.094222+0000","last_peered":"2024-09-15T21:00:31.094222+0000","last_clean":"2024-09-15T21:00:31.094222+0000","last_became_active":"2024-09-15T21:00:20.132390+0000","last_became_peered":"2024-09-15T21:00:20.132390+0000","last_unstale":"2024-09-15T21:00:31.094222+0000","last_undegraded":"2024-09-15T21:00:31.094222+0000","last_fullsized":"2024-09-15T21:00:31.094222+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:45.473402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088748799999999997,"stat_sum":{"num_bytes":2293760,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":1493,"num_write":9,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.a","version":"32'2","reported_seq":54,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.197920+0000","last_change":"2024-09-15T21:00:31.198200+0000","last_active":"2024-09-15T21:00:31.197920+0000","last_peered":"2024-09-15T21:00:31.197920+0000","last_clean":"2024-09-15T21:00:31.197920+0000","last_became_active":"2024-09-15T21:00:20.214883+0000","last_became_peered":"2024-09-15T21:00:20.214883+0000","last_unstale":"2024-09-15T21:00:31.197920+0000","last_undegraded":"2024-09-15T21:00:31.197920+0000","last_fullsized":"2024-09-15T21:00:31.197920+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:22.418043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070419700000000005,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.9","version":"39'17","reported_seq":78,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.308513+0000","last_change":"2024-09-15T21:00:31.311110+0000","last_active":"2024-09-15T21:00:31.308513+0000","last_peered":"2024-09-15T21:00:31.308513+0000","last_clean":"2024-09-15T21:00:31.308513+0000","last_became_active":"2024-09-15T21:00:20.308787+0000","last_became_peered":"2024-09-15T21:00:20.308787+0000","last_unstale":"2024-09-15T21:00:31.308513+0000","last_undegraded":"2024-09-15T21:00:31.308513+0000","last_fullsized":"2024-09-15T21:00:31.308513+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:47.270782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.211453379,"stat_sum":{"num_bytes":3850240,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":1435,"num_write":12,"num_write_kb":4112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.8","version":"33'3","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T21:00:32.407199+0000","last_change":"2024-09-15T21:00:31.307754+0000","last_active":"2024-09-15T21:00:32.407199+0000","last_peered":"2024-09-15T21:00:32.407199+0000","last_clean":"2024-09-15T21:00:32.407199+0000","last_became_active":"2024-09-15T21:00:20.486849+0000","last_became_peered":"2024-09-15T21:00:20.486849+0000","last_unstale":"2024-09-15T21:00:32.407199+0000","last_undegraded":"2024-09-15T21:00:32.407199+0000","last_fullsized":"2024-09-15T21:00:32.407199+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:40.524788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00126593,"stat_sum":{"num_bytes":180224,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":164,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.103198+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:31.103198+0000","last_peered":"2024-09-15T21:00:31.103198+0000","last_clean":"2024-09-15T21:00:31.103198+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:31.103198+0000","last_undegraded":"2024-09-15T21:00:31.103198+0000","last_fullsized":"2024-09-15T21:00:31.103198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.069378+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:31.069378+0000","last_peered":"2024-09-15T21:00:31.069378+0000","last_clean":"2024-09-15T21:00:31.069378+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:31.069378+0000","last_undegraded":"2024-09-15T21:00:31.069378+0000","last_fullsized":"2024-09-15T21:00:31.069378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"38'23","reported_seq":84,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.096580+0000","last_change":"2024-09-15T21:00:31.096580+0000","last_active":"2024-09-15T21:00:31.096580+0000","last_peered":"2024-09-15T21:00:31.096580+0000","last_clean":"2024-09-15T21:00:31.096580+0000","last_became_active":"2024-09-15T21:00:20.411551+0000","last_became_peered":"2024-09-15T21:00:20.411551+0000","last_unstale":"2024-09-15T21:00:31.096580+0000","last_undegraded":"2024-09-15T21:00:31.096580+0000","last_fullsized":"2024-09-15T21:00:31.096580+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:20.248992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024594130000000001,"stat_sum":{"num_bytes":2670592,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":8,"num_write":20,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.102946+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:31.102946+0000","last_peered":"2024-09-15T21:00:31.102946+0000","last_clean":"2024-09-15T21:00:31.102946+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:31.102946+0000","last_undegraded":"2024-09-15T21:00:31.102946+0000","last_fullsized":"2024-09-15T21:00:31.102946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"32'4","reported_seq":56,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:00:30.081383+0000","last_change":"2024-09-15T21:00:30.082986+0000","last_active":"2024-09-15T21:00:30.081383+0000","last_peered":"2024-09-15T21:00:30.081383+0000","last_clean":"2024-09-15T21:00:30.081383+0000","last_became_active":"2024-09-15T21:00:20.315752+0000","last_became_peered":"2024-09-15T21:00:20.315752+0000","last_unstale":"2024-09-15T21:00:30.081383+0000","last_undegraded":"2024-09-15T21:00:30.081383+0000","last_fullsized":"2024-09-15T21:00:30.081383+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:36.240570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022627680000000001,"stat_sum":{"num_bytes":606208,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2029,"num_write":4,"num_write_kb":592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T21:00:32.284521+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:32.284521+0000","last_peered":"2024-09-15T21:00:32.284521+0000","last_clean":"2024-09-15T21:00:32.284521+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:32.284521+0000","last_undegraded":"2024-09-15T21:00:32.284521+0000","last_fullsized":"2024-09-15T21:00:32.284521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"34'10","reported_seq":71,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-15T21:00:32.407168+0000","last_change":"2024-09-15T21:00:31.308232+0000","last_active":"2024-09-15T21:00:32.407168+0000","last_peered":"2024-09-15T21:00:32.407168+0000","last_clean":"2024-09-15T21:00:32.407168+0000","last_became_active":"2024-09-15T21:00:20.517607+0000","last_became_peered":"2024-09-15T21:00:20.517607+0000","last_unstale":"2024-09-15T21:00:32.407168+0000","last_undegraded":"2024-09-15T21:00:32.407168+0000","last_fullsized":"2024-09-15T21:00:32.407168+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:25.118493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101727743,"stat_sum":{"num_bytes":1982464,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2807,"num_write":10,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":52,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.169439+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:31.169439+0000","last_peered":"2024-09-15T21:00:31.169439+0000","last_clean":"2024-09-15T21:00:31.169439+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:31.169439+0000","last_undegraded":"2024-09-15T21:00:31.169439+0000","last_fullsized":"2024-09-15T21:00:31.169439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"38'12","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.097288+0000","last_change":"2024-09-15T21:00:31.097563+0000","last_active":"2024-09-15T21:00:31.097288+0000","last_peered":"2024-09-15T21:00:31.097288+0000","last_clean":"2024-09-15T21:00:31.097288+0000","last_became_active":"2024-09-15T21:00:20.410469+0000","last_became_peered":"2024-09-15T21:00:20.410469+0000","last_unstale":"2024-09-15T21:00:31.097288+0000","last_undegraded":"2024-09-15T21:00:31.097288+0000","last_fullsized":"2024-09-15T21:00:31.097288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:28.675179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047606800000000001,"stat_sum":{"num_bytes":1703936,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":8,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"3.5","version":"39'40","reported_seq":104,"reported_epoch":39,"state":"active+clean+snaptrim","last_fresh":"2024-09-15T21:00:31.529319+0000","last_change":"2024-09-15T21:00:31.088624+0000","last_active":"2024-09-15T21:00:31.529319+0000","last_peered":"2024-09-15T21:00:31.529319+0000","last_clean":"2024-09-15T21:00:31.529319+0000","last_became_active":"2024-09-15T21:00:20.585838+0000","last_became_peered":"2024-09-15T21:00:20.585838+0000","last_unstale":"2024-09-15T21:00:31.529319+0000","last_undegraded":"2024-09-15T21:00:31.529319+0000","last_fullsized":"2024-09-15T21:00:31.529319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:37.825173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71189026700000002,"stat_sum":{"num_bytes":7045120,"num_objects":13,"num_object_clones":5,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":33,"num_read_kb":2176,"num_write":28,"num_write_kb":6880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:00:30.277084+0000","last_change":"2024-09-15T21:00:07.903038+0000","last_active":"2024-09-15T21:00:30.277084+0000","last_peered":"2024-09-15T21:00:30.277084+0000","last_clean":"2024-09-15T21:00:30.277084+0000","last_became_active":"2024-09-15T21:00:06.995916+0000","last_became_peered":"2024-09-15T21:00:06.995916+0000","last_unstale":"2024-09-15T21:00:30.277084+0000","last_undegraded":"2024-09-15T21:00:30.277084+0000","last_fullsized":"2024-09-15T21:00:30.277084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:17.651038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"38'25","reported_seq":75,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:00:30.583012+0000","last_change":"2024-09-15T21:00:30.585183+0000","last_active":"2024-09-15T21:00:30.583012+0000","last_peered":"2024-09-15T21:00:30.583012+0000","last_clean":"2024-09-15T21:00:30.583012+0000","last_became_active":"2024-09-15T21:00:20.492192+0000","last_became_peered":"2024-09-15T21:00:20.492192+0000","last_unstale":"2024-09-15T21:00:30.583012+0000","last_undegraded":"2024-09-15T21:00:30.583012+0000","last_fullsized":"2024-09-15T21:00:30.583012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:38.684625+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.30858175799999998,"stat_sum":{"num_bytes":3293184,"num_objects":8,"num_object_clones":1,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":19,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.099405+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:31.099405+0000","last_peered":"2024-09-15T21:00:31.099405+0000","last_clean":"2024-09-15T21:00:31.099405+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:31.099405+0000","last_undegraded":"2024-09-15T21:00:31.099405+0000","last_fullsized":"2024-09-15T21:00:31.099405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"34'5","reported_seq":59,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.095201+0000","last_change":"2024-09-15T21:00:31.096524+0000","last_active":"2024-09-15T21:00:31.095201+0000","last_peered":"2024-09-15T21:00:31.095201+0000","last_clean":"2024-09-15T21:00:31.095201+0000","last_became_active":"2024-09-15T21:00:20.313299+0000","last_became_peered":"2024-09-15T21:00:20.313299+0000","last_unstale":"2024-09-15T21:00:31.095201+0000","last_undegraded":"2024-09-15T21:00:31.095201+0000","last_fullsized":"2024-09-15T21:00:31.095201+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:36.813119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020873110000000001,"stat_sum":{"num_bytes":1490944,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"4","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:00:30.189401+0000","last_change":"2024-09-15T21:00:07.902991+0000","last_active":"2024-09-15T21:00:30.189401+0000","last_peered":"2024-09-15T21:00:30.189401+0000","last_clean":"2024-09-15T21:00:30.189401+0000","last_became_active":"2024-09-15T21:00:05.988808+0000","last_became_peered":"2024-09-15T21:00:05.988808+0000","last_unstale":"2024-09-15T21:00:30.189401+0000","last_undegraded":"2024-09-15T21:00:30.189401+0000","last_fullsized":"2024-09-15T21:00:30.189401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:08.990556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135839,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"34'7","reported_seq":58,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:00:30.081561+0000","last_change":"2024-09-15T21:00:30.086159+0000","last_active":"2024-09-15T21:00:30.081561+0000","last_peered":"2024-09-15T21:00:30.081561+0000","last_clean":"2024-09-15T21:00:30.081561+0000","last_became_active":"2024-09-15T21:00:20.416420+0000","last_became_peered":"2024-09-15T21:00:20.416420+0000","last_unstale":"2024-09-15T21:00:30.081561+0000","last_undegraded":"2024-09-15T21:00:30.081561+0000","last_fullsized":"2024-09-15T21:00:30.081561+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:20.760204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051628239999999999,"stat_sum":{"num_bytes":851968,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":322,"num_write":26,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-15T21:00:31.096561+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:31.096561+0000","last_peered":"2024-09-15T21:00:31.096561+0000","last_clean":"2024-09-15T21:00:31.096561+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:31.096561+0000","last_undegraded":"2024-09-15T21:00:31.096561+0000","last_fullsized":"2024-09-15T21:00:31.096561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":29589504,"num_objects":58,"num_object_clones":11,"num_object_copies":464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":2,"num_read":154,"num_read_kb":10969,"num_write":183,"num_write_kb":30288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62521344,"data_stored":71176192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378635,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30932,"kb_used_data":3908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340908,"statfs":{"total":96636764160,"available":96605089792,"internally_reserved":0,"allocated":4001792,"data_stored":4236493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411339,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38856,"kb_used_data":7660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332984,"statfs":{"total":96636764160,"available":96596975616,"internally_reserved":0,"allocated":7843840,"data_stored":8656412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34320,"kb_used_data":7292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337520,"statfs":{"total":96636764160,"available":96601620480,"internally_reserved":0,"allocated":7467008,"data_stored":8541180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411337,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30720,"kb_used_data":3696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341120,"statfs":{"total":96636764160,"available":96605306880,"internally_reserved":0,"allocated":3784704,"data_stored":3793651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411337,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":5423104,"data_stored":5584147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":5427200,"data_stored":5747443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444043,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":5775360,"data_stored":6083315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":6308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":6459392,"data_stored":7119887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,9,8,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444041,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31232,"kb_used_data":4200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340608,"statfs":{"total":96636764160,"available":96604782592,"internally_reserved":0,"allocated":4300800,"data_stored":4461299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444043,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":4884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":5001216,"data_stored":5186291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444041,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33540,"kb_used_data":6512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338300,"statfs":{"total":96636764160,"available":96602419200,"internally_reserved":0,"allocated":6668288,"data_stored":7209485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31148,"kb_used_data":4120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340692,"statfs":{"total":96636764160,"available":96604868608,"internally_reserved":0,"allocated":4218880,"data_stored":4456954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3821568,"data_stored":4255744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6324224,"data_stored":7008256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3567616,"data_stored":4009984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5009408,"data_stored":5636096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3891200,"data_stored":4423680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":7655424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6037504,"data_stored":6787072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":5357568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":3375104,"data_stored":3756032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5435392,"data_stored":6127616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":8269824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":6889472,"data_stored":7888896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:35.453 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:35.453 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:35.453 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:35.454 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:35.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:35.489 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:35.489+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:35.591 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:35.587+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:35.633 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:35.633 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:35.633 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:35.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:35.691 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:35.689+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:35.778 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:35.778 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:35.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:35.778 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 32/28 ECSubRead(tid=25, to_read={3:a5da2594:::smithi02732947-25:head=0,4096,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.610369436999999, 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10096388000000001, 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163324+0000", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163325+0000", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8199999999999999e-07 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163332+0000", 2024-09-15T21:00:35.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1960000000000002e-06 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163354+0000", 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2116e-05 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163372+0000", 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8238000000000001e-05 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263745+0000", 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100372425 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264288+0000", 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054372299999999997 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 33/28 ECSubRead(tid=15, to_read={3:2f703215:::smithi02732947-30:head=0,4096,0}, subchunks={3:2f703215:::smithi02732947-30:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:35.782 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.493778828, 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10090344699999999, 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279915+0000", 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279917+0000", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.897e-06 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279934+0000", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7198e-05 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279967+0000", 2024-09-15T21:00:35.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3197000000000001e-05 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.279991+0000", 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3487000000000001e-05 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380280+0000", 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100288711 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380819+0000", 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053895699999999996 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.493645098, 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134782000000001, 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.786 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280049+0000", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280050+0000", 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:00:35.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280059+0000", 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4259999999999992e-06 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280077+0000", 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7948000000000002e-05 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280091+0000", 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4338999999999999e-05 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380947+0000", 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100855271 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381397+0000", 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044987 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=17, reqid=client.4641.0:106, at_version=33'11, trim_to=0'0, roll_forward_to=32'10))", 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.493557864, 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103549026, 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280137+0000", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280251+0000", 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000113705 2024-09-15T21:00:35.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280273+0000", 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2198e-05 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280285+0000", 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2014999999999999e-05 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381500+0000", 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10121572400000001 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381526+0000", 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5077000000000001e-05 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383596+0000", 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020703029999999999 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383685+0000", 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9223999999999998e-05 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=29, reqid=client.4641.0:124, at_version=33'11, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.196321027, 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103593675, 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577375+0000", 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9870000000000002e-06 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577857+0000", 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048242599999999999 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577888+0000", 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1007000000000003e-05 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577904+0000", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5588000000000001e-05 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578065+0000", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000161593 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578111+0000", 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5977999999999999e-05 2024-09-15T21:00:35.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680842+0000", 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102730619 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680967+0000", 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012447700000000001 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.798 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=30, reqid=client.4641.0:125, at_version=33'12, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.195749656, 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10310723500000001, 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577945+0000", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.096e-06 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577980+0000", 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4836000000000003e-05 2024-09-15T21:00:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577999+0000", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8975e-05 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578040+0000", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1328999999999997e-05 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579237+0000", 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011965879999999999 2024-09-15T21:00:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579269+0000", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.222e-05 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680983+0000", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10171382499999999 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681051+0000", 2024-09-15T21:00:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8366000000000002e-05 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=31, reqid=client.4641.0:130, at_version=33'13, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.195622443, 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10373313300000001, 2024-09-15T21:00:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578073+0000", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.319e-06 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578381+0000", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000308282 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578400+0000", 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8788e-05 2024-09-15T21:00:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578412+0000", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2509e-05 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679722+0000", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10130917 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679753+0000", 2024-09-15T21:00:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1386000000000001e-05 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681744+0000", 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019913180000000002 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681805+0000", 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0361e-05 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.178142620999999, 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:00:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:00:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:00:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.690190232, 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:00:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:00:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.583073+0000", 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.1906204320000002, 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10257732899999999, 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583073+0000", 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583073+0000", 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583074+0000", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3399999999999998e-07 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583090+0000", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.626e-05 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583102+0000", 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1514e-05 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.583108+0000", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9329999999999998e-06 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.682032+0000", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.098924148000000003 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.682043+0000", 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0796e-05 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.685613+0000", 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035697440000000001 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.685651+0000", 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.82e-05 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9871766019999999, 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:00:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:00:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/28 ECSubWrite(tid=76, reqid=client.4641.0:244, at_version=39'36, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.102784+0000", 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.6709102510000005, 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10275662000000001, 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.102784+0000", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.102784+0000", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.102784+0000", 2024-09-15T21:00:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.103184+0000", 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00039957300000000002 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.103213+0000", 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8787000000000001e-05 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.103241+0000", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8308000000000001e-05 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.103360+0000", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118893 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.103370+0000", 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0162e-05 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.205516+0000", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102146313 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.205540+0000", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3944e-05 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 39/28 ECSubWrite(tid=77, reqid=osd.5.0:77, at_version=39'38, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.107140+0000", 2024-09-15T21:00:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.6665543469999999, 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099145598000000001, 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107140+0000", 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107140+0000", 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107141+0000", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107195+0000", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3788000000000001e-05 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107244+0000", 2024-09-15T21:00:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8749999999999999e-05 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.107269+0000", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4561000000000001e-05 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.205281+0000", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.098012441000000006 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.205288+0000", 2024-09-15T21:00:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3869999999999997e-06 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.206259+0000", 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00097109400000000004 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.206285+0000", 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5687e-05 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 39/28 ECSubWrite(tid=37, reqid=client.4641.0:246, at_version=39'26, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.110913+0000", 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.662780744, 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101979021, 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.110913+0000", 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.110913+0000", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.110916+0000", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6800000000000002e-06 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.111326+0000", 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040996799999999998 2024-09-15T21:00:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.111372+0000", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5713000000000001e-05 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.211462+0000", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10009095 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.211582+0000", 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000119646 2024-09-15T21:00:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.211596+0000", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3434e-05 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.212802+0000", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001206555 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.212892+0000", 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0074999999999994e-05 2024-09-15T21:00:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=79, reqid=osd.5.0:79, at_version=39'14, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.4616043220000003, 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103073317, 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7000000000000005e-07 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312125+0000", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4603000000000001e-05 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312135+0000", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.04e-05 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312143+0000", 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3950000000000003e-06 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312254+0000", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111611 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312269+0000", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4788e-05 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415084+0000", 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102815383 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415163+0000", 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8566999999999995e-05 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=80, reqid=client.4641.0:250, at_version=39'15, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.4615331459999998, 2024-09-15T21:00:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10304223699999999, 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312378+0000", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021675199999999999 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312400+0000", 2024-09-15T21:00:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2379e-05 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312409+0000", 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0939999999999997e-06 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.412940+0000", 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10053021500000001 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.412958+0000", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8535000000000001e-05 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415170+0000", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022119370000000002 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415203+0000", 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2947000000000002e-05 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=81, reqid=client.4641.0:251, at_version=39'16, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.4612671390000003, 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102905732, 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312521+0000", 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3363999999999996e-05 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312533+0000", 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2804e-05 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312542+0000", 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0149999999999992e-06 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.413828+0000", 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101285968 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.413844+0000", 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5849e-05 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415302+0000", 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014583420000000001 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415332+0000", 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9977000000000001e-05 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=82, reqid=client.4641.0:252, at_version=39'17, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.4611375779999998, 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103356555, 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312557+0000", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4900000000000001e-07 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312587+0000", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0343000000000001e-05 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312596+0000", 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312602+0000", 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3459999999999999e-06 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.414704+0000", 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102101729 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.414731+0000", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7211000000000001e-05 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415849+0000", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011185909999999999 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415913+0000", 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3417999999999995e-05 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.4010533040000004, 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104322594, 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372642+0000", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7069999999999999e-06 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372856+0000", 2024-09-15T21:00:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000213255 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372963+0000", 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107661 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473110+0000", 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100146978 2024-09-15T21:00:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473309+0000", 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019897599999999999 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473331+0000", 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1427999999999999e-05 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476903+0000", 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035721609999999999 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476963+0000", 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0427999999999998e-05 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:00:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.473176+0000", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.3005176550000002, 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101151476, 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473176+0000", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473176+0000", 2024-09-15T21:00:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473178+0000", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.313e-06 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473210+0000", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2957000000000003e-05 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473236+0000", 2024-09-15T21:00:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5494e-05 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473256+0000", 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9732e-05 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.573438+0000", 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10018210299999999 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.573448+0000", 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0246e-05 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.574283+0000", 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00083510300000000002 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.574328+0000", 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4527999999999998e-05 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.869 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:35.870 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:35.870 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:35.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:35.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:35.869+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:35.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:35.967+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:36.042 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:36.043 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:36.043 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:36.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:36.102 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:36.101+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:36.103 INFO:tasks.rados.rados.0.smithi027.stdout:112: done (1 left) 2024-09-15T21:00:36.103 INFO:tasks.rados.rados.0.smithi027.stdout:114: done (0 left) 2024-09-15T21:00:36.103 INFO:tasks.rados.rados.0.smithi027.stdout:115: read oid 42 snap -1 2024-09-15T21:00:36.103 INFO:tasks.rados.rados.0.smithi027.stdout:115: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T21:00:36.103 INFO:tasks.rados.rados.0.smithi027.stdout:116: setattr oid 24 current snap is 7 2024-09-15T21:00:36.104 INFO:tasks.rados.rados.0.smithi027.stdout:117: setattr oid 16 current snap is 7 2024-09-15T21:00:36.106 INFO:tasks.rados.rados.0.smithi027.stdout:118: snap_remove snap 1 2024-09-15T21:00:36.174 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:36.173+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:36.178 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:36.178 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:36.178 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:36.192 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:36.286 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:36.285+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:36.314 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.307+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:36.314 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.307+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:36.314 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.307+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:36.314 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.307+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,4,5 2024-09-15T21:00:36.314 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.307+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:36.367 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.389171425000001, 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:00:36.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.369 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:00:36.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.38716018, 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:00:36.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:00:36.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:00:36.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:00:36.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386408526, 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:00:36.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.386342342000001, 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:00:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:128 3.7s0 3:e8b6cabe:::smithi02732947-43:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.187422418000001, 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51140068500000002, 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 128 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178182+0000", 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178183+0000", 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:00:36.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178189+0000", 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8479999999999997e-06 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.178216+0000", 2024-09-15T21:00:36.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6353e-05 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278315+0000", 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100099326 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.365033+0000", 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.086718609000000002 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.365192+0000", 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158603 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366018+0000", 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00082617400000000005 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367592+0000", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001574085 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689318+0000", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32172605500000001 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689583+0000", 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026455899999999999 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.403 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:129 3.7s0 3:e8b6cabe:::smithi02732947-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.087237056999999, 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61157217500000005, 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 129 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:36.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278370+0000", 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6129999999999998e-06 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278383+0000", 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3026e-05 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278414+0000", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0857e-05 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278434+0000", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9924000000000001e-05 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366928+0000", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088493789000000003 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:36.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367150+0000", 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000222081 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690379+0000", 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32322944100000001 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690606+0000", 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022645599999999999 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.889940+0000", 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19933398799999999 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.881081517, 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:36.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:00:36.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.879644567, 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:00:36.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.879566422, 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:00:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:00:36.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.777051288999999, 2024-09-15T21:00:36.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:00:36.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.697004562, 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.430 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:00:36.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.682616471999999, 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.434 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:36.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:00:36.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:00:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:00:36.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:00:36.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.676206517000001, 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.442 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:00:36.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.58868045, 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51692596999999996, 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776924+0000", 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776926+0000", 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5799999999999999e-06 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776946+0000", 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0315000000000001e-05 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776971+0000", 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4706999999999999e-05 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.776989+0000", 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8422000000000001e-05 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293605+0000", 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51661528099999998 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293850+0000", 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245665 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.451 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.577558754, 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60734763000000003, 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788048+0000", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.063e-06 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788069+0000", 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1321000000000001e-05 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788097+0000", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8136999999999998e-05 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888193+0000", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100095434 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394534+0000", 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50634117999999995 2024-09-15T21:00:36.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395394+0000", 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00085949500000000005 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:148 3.bs0 3:d37a3ed4:::smithi02732947-49:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.575715731000001, 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60489703100000003, 2024-09-15T21:00:36.455 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 148 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789889+0000", 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789891+0000", 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789902+0000", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1362000000000001e-05 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789921+0000", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8868e-05 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789935+0000", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4059e-05 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.793875+0000", 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039398139999999998 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.794018+0000", 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000143635 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.794823+0000", 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080479900000000003 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795864+0000", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010403669999999999 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394660+0000", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.598796632 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394786+0000", 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012575800000000001 2024-09-15T21:00:36.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.575653544, 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:00:36.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:00:36.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.493696167, 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62454644199999998, 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.466 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871910+0000", 2024-09-15T21:00:36.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.55e-06 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871958+0000", 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7698000000000003e-05 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871989+0000", 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1061e-05 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.872007+0000", 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7934000000000001e-05 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496399+0000", 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62439274499999997 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496455+0000", 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5454000000000001e-05 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.476828727000001, 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60786671999999997, 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888777+0000", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4e-07 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888795+0000", 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8403999999999999e-05 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888808+0000", 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2359e-05 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888816+0000", 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3359999999999995e-06 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496583+0000", 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60776654399999996 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496643+0000", 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0136999999999997e-05 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.473 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:160 3.2s0 3:48a96439:::smithi02732947-3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e33)", 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.072588425999999, 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61085863699999998, 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 160 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:36.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293018+0000", 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.074e-06 2024-09-15T21:00:36.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293027+0000", 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3820000000000004e-06 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293049+0000", 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2697999999999999e-05 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293072+0000", 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671e-05 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699705+0000", 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406633417 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699975+0000", 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026989499999999998 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.701228+0000", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012527560000000001 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.702476+0000", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001248033 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903693+0000", 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20121672600000001 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:36.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903875+0000", 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018198500000000001 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:36.479 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:36.481 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:36.481 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:36.477+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:36.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.507+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:36.513 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.507+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,4,5 2024-09-15T21:00:36.581 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.575+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:36.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.611+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:36.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.611+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6 to read 0,1,4,5 2024-09-15T21:00:36.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:36.611+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:00:36.622 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:36.622 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:36.622 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:36.637 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:36.723 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:36.719+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:36.782 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:36.781+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:36.800 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:36.800 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:36.800 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:36.801 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:36.801 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:36.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:36.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:36.884 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:36.881+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:36.976 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:36.976 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:36.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:36.979+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.215532025, 2024-09-15T21:00:36.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:36.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:00:36.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.214430659, 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:36.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:36.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:36.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:00:37.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:00:37.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:37.002 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.214390155, 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:37.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:00:37.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.007 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.193838646, 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:37.008 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:00:37.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:00:37.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:00:37.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.125551048, 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:00:37.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.016 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.092531790000001, 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:37.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.022 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.0924914, 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:00:37.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.027 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.024279235, 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.031 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.02237643, 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:00:37.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.020318101000001, 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202146044, 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:37.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965234+0000", 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7099999999999999e-06 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965259+0000", 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5732999999999999e-05 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965292+0000", 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2223e-05 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965310+0000", 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8087000000000001e-05 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167355+0000", 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202045846 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167378+0000", 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2444999999999999e-05 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.039 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.010394363, 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:00:37.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.009584623, 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.909652918999999, 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:00:37.053 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.889976888, 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.054 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:37.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:00:37.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:00:37.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:00:37.060 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.060 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=6, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.605869024, 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19803510199999999, 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379683+0000", 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.917e-06 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379693+0000", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.01e-05 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379722+0000", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8623e-05 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379740+0000", 2024-09-15T21:00:37.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8677000000000001e-05 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577690+0000", 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19795005800000001 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577716+0000", 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5726999999999999e-05 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.598535173, 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:00:37.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.401112106999999, 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.073 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=18, reqid=client.4641.0:143, at_version=33'4, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.396726807, 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013446600000001, 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588825+0000", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588950+0000", 2024-09-15T21:00:37.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124919 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588978+0000", 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8500000000000002e-05 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588993+0000", 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5237e-05 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785673+0000", 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19668001800000001 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785700+0000", 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7112999999999999e-05 2024-09-15T21:00:37.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788921+0000", 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032205490000000001 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788958+0000", 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6619999999999998e-05 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.078 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.317876299, 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:37.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:00:37.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 33/28 ECSubReadReply(tid=8, attrs_read=0))", 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.298677490999999, 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19621026899999999, 2024-09-15T21:00:37.083 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686872+0000", 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686874+0000", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7999999999999999e-06 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686892+0000", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7258e-05 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686916+0000", 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4091999999999999e-05 2024-09-15T21:00:37.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.686928+0000", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2242000000000001e-05 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.883066+0000", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19613826500000001 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.883083+0000", 2024-09-15T21:00:37.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6612e-05 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:37.087 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:37.088 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:37.088 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:37.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:37.089 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:37.090 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:37.090 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:37.091 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:37.091 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:37.091 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:37.091 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:37.092 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:37.092 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:37.092 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:37.085+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:37.105 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:37.105 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:37.111 INFO:tasks.rados.rados.0.smithi027.stdout:118: done (3 left) 2024-09-15T21:00:37.111 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 23 current snap is 7 2024-09-15T21:00:37.112 INFO:tasks.rados.rados.0.smithi027.stdout:119: seq_num 70 ranges {753664=802816,1556480=376832} 2024-09-15T21:00:37.122 INFO:tasks.rados.rados.0.smithi027.stdout:119: writing smithi02732947-23 from 753664 to 1556480 tid 1 2024-09-15T21:00:37.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:00:37.124 INFO:tasks.rados.rados.0.smithi027.stdout:119: writing smithi02732947-23 from 1556480 to 1933312 tid 2 2024-09-15T21:00:37.124 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 49 current snap is 7 2024-09-15T21:00:37.124 INFO:tasks.rados.rados.0.smithi027.stdout:120: seq_num 71 ranges {819200=425984,1245184=573440,1818624=196608} 2024-09-15T21:00:37.131 INFO:tasks.rados.rados.0.smithi027.stdout:120: writing smithi02732947-49 from 819200 to 1245184 tid 1 2024-09-15T21:00:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:37.133 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:37.134 INFO:tasks.rados.rados.0.smithi027.stdout:120: writing smithi02732947-49 from 1245184 to 1818624 tid 2 2024-09-15T21:00:37.136 INFO:tasks.rados.rados.0.smithi027.stdout:120: writing smithi02732947-49 from 1818624 to 2015232 tid 3 2024-09-15T21:00:37.136 INFO:tasks.rados.rados.0.smithi027.stdout:121: snap_remove snap 4 2024-09-15T21:00:37.153 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:37.153 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:37.173 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:37.174 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:37.189 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:37.185+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:37.243 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:37.269 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:37.269 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:37.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:37.330 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:37.327+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:37.390 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:37.389+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:37.413 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:37.414 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:37.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:37.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:37.515+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:37.592 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:37.589+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:37.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:37.694 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:37.693+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:37.778 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:37.778 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:37.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:37.794 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:37.793+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:37.895 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:37.891+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:37.920 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:37.920 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:37.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:38.023 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:38.019+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:38.097 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:38.091+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:38.098 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.098 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.113 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:00:38.132 INFO:tasks.rados.rados.0.smithi027.stdout:121: done (5 left) 2024-09-15T21:00:38.132 INFO:tasks.rados.rados.0.smithi027.stdout:122: delete oid 45 current snap is 7 2024-09-15T21:00:38.132 INFO:tasks.rados.rados.0.smithi027.stdout:115: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T21:00:38.132 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 44227 2024-09-15T21:00:38.134 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.133+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:38.135 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.133+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:38.135 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.133+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5 to read 0,1,4,5 2024-09-15T21:00:38.137 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 150021 2024-09-15T21:00:38.139 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 24044 2024-09-15T21:00:38.143 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 24 v 19 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T21:00:38.143 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 47 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-09-15T21:00:38.143 INFO:tasks.rados.rados.0.smithi027.stdout:119: finishing write tid 1 to smithi02732947-23 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:119: finishing write tid 2 to smithi02732947-23 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:120: finishing write tid 1 to smithi02732947-49 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:120: finishing write tid 2 to smithi02732947-49 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:120: finishing write tid 3 to smithi02732947-49 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:120: finishing write tid 4 to smithi02732947-49 2024-09-15T21:00:38.144 INFO:tasks.rados.rados.0.smithi027.stdout:119: finishing write tid 3 to smithi02732947-23 2024-09-15T21:00:38.202 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.201+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:38.236 INFO:tasks.rados.rados.0.smithi027.stdout:120: finishing write tid 5 to smithi02732947-49 2024-09-15T21:00:38.236 INFO:tasks.rados.rados.0.smithi027.stdout:120: oid 49 updating version 0 to 26 2024-09-15T21:00:38.236 INFO:tasks.rados.rados.0.smithi027.stdout:120: oid 49 updating version 26 to 27 2024-09-15T21:00:38.236 INFO:tasks.rados.rados.0.smithi027.stdout:120: oid 49 updating version 27 to 29 2024-09-15T21:00:38.237 INFO:tasks.rados.rados.0.smithi027.stdout:120: oid 49 version 29 is already newer than 28 2024-09-15T21:00:38.237 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 49 v 29 (ObjNum 71 snap 7 seq_num 71) dirty exists 2024-09-15T21:00:38.237 INFO:tasks.rados.rados.0.smithi027.stdout:120: left oid 49 (ObjNum 71 snap 7 seq_num 71) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:115: done (5 left) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:116: done (4 left) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:117: done (3 left) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:120: done (2 left) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:122: done (1 left) 2024-09-15T21:00:38.242 INFO:tasks.rados.rados.0.smithi027.stdout:123: snap_create 2024-09-15T21:00:38.242 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:38.241+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:38.243 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:38.241+0000 7fbc06107640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:38.244 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:38.241+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:00:38.244 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:38.241+0000 7fbc0a10f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:00:38.244 INFO:tasks.rados.rados.0.smithi027.stdout:119: finishing write tid 4 to smithi02732947-23 2024-09-15T21:00:38.244 INFO:tasks.rados.rados.0.smithi027.stdout:119: oid 23 updating version 0 to 53 2024-09-15T21:00:38.245 INFO:tasks.rados.rados.0.smithi027.stdout:119: oid 23 version 53 is already newer than 51 2024-09-15T21:00:38.245 INFO:tasks.rados.rados.0.smithi027.stdout:119: oid 23 version 53 is already newer than 52 2024-09-15T21:00:38.245 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 23 v 53 (ObjNum 70 snap 7 seq_num 70) dirty exists 2024-09-15T21:00:38.245 INFO:tasks.rados.rados.0.smithi027.stdout:119: left oid 23 (ObjNum 70 snap 7 seq_num 70) 2024-09-15T21:00:38.280 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.281 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:38.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:38.295+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:38.400 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:38.397+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:38.423 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.423 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:38.501 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:38.495+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:38.533 INFO:tasks.rados.rados.0.smithi027.stdout:119: done (1 left) 2024-09-15T21:00:38.534 INFO:tasks.rados.rados.0.smithi027.stdout:123: done (0 left) 2024-09-15T21:00:38.534 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 31 current snap is 8 2024-09-15T21:00:38.534 INFO:tasks.rados.rados.0.smithi027.stdout:124: seq_num 73 ranges {507904=458752,966656=475136,1441792=294912} 2024-09-15T21:00:38.550 INFO:tasks.rados.rados.0.smithi027.stdout:124: writing smithi02732947-31 from 507904 to 966656 tid 1 2024-09-15T21:00:38.559 INFO:tasks.rados.rados.0.smithi027.stdout:124: writing smithi02732947-31 from 966656 to 1441792 tid 2 2024-09-15T21:00:38.564 INFO:tasks.rados.rados.0.smithi027.stdout:124: writing smithi02732947-31 from 1441792 to 1736704 tid 3 2024-09-15T21:00:38.565 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 16 current snap is 8 2024-09-15T21:00:38.565 INFO:tasks.rados.rados.0.smithi027.stdout:125: seq_num 74 ranges {1228800=475136,1703936=622592,2326528=147456} 2024-09-15T21:00:38.570 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.570 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:38.590 INFO:tasks.rados.rados.0.smithi027.stdout:125: writing smithi02732947-16 from 1228800 to 1703936 tid 1 2024-09-15T21:00:38.598 INFO:tasks.rados.rados.0.smithi027.stdout:125: writing smithi02732947-16 from 1703936 to 2326528 tid 2 2024-09-15T21:00:38.600 INFO:tasks.rados.rados.0.smithi027.stdout:125: writing smithi02732947-16 from 2326528 to 2473984 tid 3 2024-09-15T21:00:38.600 INFO:tasks.rados.rados.0.smithi027.stdout:126: copy_from oid 10 from oid 39 current snap is 8 2024-09-15T21:00:38.600 INFO:tasks.rados.rados.0.smithi027.stdout:127: read oid 48 snap 7 2024-09-15T21:00:38.600 INFO:tasks.rados.rados.0.smithi027.stdout:127: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-15T21:00:38.601 INFO:tasks.rados.rados.0.smithi027.stdout:128: delete oid 12 current snap is 8 2024-09-15T21:00:38.627 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:38.623+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:38.703 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:38.701+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:38.719 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.719 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:38.804 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:38.799+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:38.808 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:38.805+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:38.808 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:38.805+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:38.857 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.853+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:38.857 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.853+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:38.857 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.853+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,4,5 2024-09-15T21:00:38.857 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:38.853+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,4,5 2024-09-15T21:00:38.905 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:38.905 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:38.905 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:38.901+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:38.910 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:38.909+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:00:38.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:38.967 INFO:tasks.rados.rados.0.smithi027.stdout:124: finishing write tid 1 to smithi02732947-31 2024-09-15T21:00:38.970 INFO:tasks.rados.rados.0.smithi027.stdout:126: finishing copy_from racing read to smithi02732947-10 2024-09-15T21:00:39.006 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:39.005+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:39.010 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.009+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:39.052 INFO:tasks.rados.rados.0.smithi027.stdout:125: finishing write tid 1 to smithi02732947-16 2024-09-15T21:00:39.069 INFO:tasks.rados.rados.0.smithi027.stdout:124: finishing write tid 2 to smithi02732947-31 2024-09-15T21:00:39.069 INFO:tasks.rados.rados.0.smithi027.stdout:124: finishing write tid 3 to smithi02732947-31 2024-09-15T21:00:39.084 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:39.084 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:39.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:39.107 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:39.105+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:39.110 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.109+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:39.110 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.109+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:00:39.152 INFO:tasks.rados.rados.0.smithi027.stdout:125: finishing write tid 2 to smithi02732947-16 2024-09-15T21:00:39.170 INFO:tasks.rados.rados.0.smithi027.stdout:126: finishing copy_from to smithi02732947-10 2024-09-15T21:00:39.170 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 10 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T21:00:39.170 INFO:tasks.rados.rados.0.smithi027.stdout:124: finishing write tid 4 to smithi02732947-31 2024-09-15T21:00:39.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:39.203+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:39.224 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:39.225 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:39.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:39.253 INFO:tasks.rados.rados.0.smithi027.stdout:125: finishing write tid 3 to smithi02732947-16 2024-09-15T21:00:39.253 INFO:tasks.rados.rados.0.smithi027.stdout:125: finishing write tid 4 to smithi02732947-16 2024-09-15T21:00:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:39.251+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:39.251+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:39.251+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:00:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:125: finishing write tid 5 to smithi02732947-16 2024-09-15T21:00:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:125: oid 16 updating version 0 to 65 2024-09-15T21:00:39.258 INFO:tasks.rados.rados.0.smithi027.stdout:125: oid 16 updating version 65 to 66 2024-09-15T21:00:39.258 INFO:tasks.rados.rados.0.smithi027.stdout:125: oid 16 version 66 is already newer than 63 2024-09-15T21:00:39.258 INFO:tasks.rados.rados.0.smithi027.stdout:125: oid 16 version 66 is already newer than 64 2024-09-15T21:00:39.258 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 66 (ObjNum 74 snap 8 seq_num 74) dirty exists 2024-09-15T21:00:39.258 INFO:tasks.rados.rados.0.smithi027.stdout:125: left oid 16 (ObjNum 74 snap 8 seq_num 74) 2024-09-15T21:00:39.259 INFO:tasks.rados.rados.0.smithi027.stdout:125: done (4 left) 2024-09-15T21:00:39.259 INFO:tasks.rados.rados.0.smithi027.stdout:126: done (3 left) 2024-09-15T21:00:39.259 INFO:tasks.rados.rados.0.smithi027.stdout:128: done (2 left) 2024-09-15T21:00:39.259 INFO:tasks.rados.rados.0.smithi027.stdout:129: snap_remove snap 5 2024-09-15T21:00:39.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:39.312 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.309+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,4,5 2024-09-15T21:00:39.313 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:39.309+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:39.406 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:39.407 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:39.410 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:39.403+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:39.415 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.413+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:39.415 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.413+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:39.415 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.413+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:00:39.415 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.413+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:00:39.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:39.430 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:39.430 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:39.447 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:39.464 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:39.464 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:39.473 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:39.469+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:39.473 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:39.469+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:39.481 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:39.482 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:39.503 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:39.503 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:39.513 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:39.509+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:39.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:39.525 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:39.547 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:39.547 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:39.569 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:39.569 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:39.573 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:39.569+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:00:39.573 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:39.569+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,7 to read 0,1,4,5 2024-09-15T21:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:39.591 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:39.591 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:39.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:39.614 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:39.614 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:39.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:39.638 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:39.637+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:39.717 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:39.717 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:39.717 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:39.718 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:39.715+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:39.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:39.819 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:39.817+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:39.897 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:39.897 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:39.897 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:39.897 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:39.897 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:39.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:39.958 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:39.957+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:40.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.019+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/28 ECSubWrite(tid=28, reqid=client.4641.0:122, at_version=33'10, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.476669+0000", 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.56004351, 2024-09-15T21:00:40.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10279836000000001, 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.476669+0000", 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.476669+0000", 2024-09-15T21:00:40.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.476670+0000", 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.476692+0000", 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1994999999999999e-05 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.476708+0000", 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5648999999999999e-05 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.576798+0000", 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100090318 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.576995+0000", 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019660700000000001 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.577012+0000", 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7166000000000001e-05 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.579410+0000", 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023983289999999998 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.579467+0000", 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7312999999999998e-05 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.8s6 33/28 ECSubWrite(tid=22, reqid=client.4641.0:154, at_version=33'3, trim_to=0'0, roll_forward_to=30'2))", 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.723787582, 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20195380099999999, 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312927+0000", 2024-09-15T21:00:40.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3e-06 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313031+0000", 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104053 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313061+0000", 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0015e-05 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313085+0000", 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3127999999999999e-05 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.513445+0000", 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20036016600000001 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.513463+0000", 2024-09-15T21:00:40.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8499e-05 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.514821+0000", 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013581699999999999 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.514879+0000", 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7469999999999997e-05 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/28 ECSubWrite(tid=39, reqid=client.4641.0:152, at_version=33'17, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.023440425, 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20457245199999999, 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:40.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013273+0000", 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4799999999999997e-07 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013291+0000", 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7527e-05 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013303+0000", 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2486e-05 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116075+0000", 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102772293 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216396+0000", 2024-09-15T21:00:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100321059 2024-09-15T21:00:40.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216405+0000", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8359999999999998e-06 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217565+0000", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001159481 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217845+0000", 2024-09-15T21:00:40.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027992199999999999 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/28 ECSubWrite(tid=40, reqid=client.4641.0:156, at_version=33'18, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.920592835000001, 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20296982799999999, 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116121+0000", 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116221+0000", 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010067 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116235+0000", 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4117000000000001e-05 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216815+0000", 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100580146 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316782+0000", 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099966606 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316815+0000", 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3225000000000001e-05 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319037+0000", 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022215490000000002 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319090+0000", 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s7 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.920400990999999, 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20252270999999999, 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116482+0000", 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017011500000000001 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116500+0000", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708999999999999e-05 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217229+0000", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100729376 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316421+0000", 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099191814000000003 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316431+0000", 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7319999999999993e-06 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318772+0000", 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023414769999999998 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318834+0000", 2024-09-15T21:00:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2162999999999994e-05 2024-09-15T21:00:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:29.580557+0000", 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.456155836000001, 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10348199700000001, 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580557+0000", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580557+0000", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580559+0000", 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.243e-06 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580595+0000", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6412000000000003e-05 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580636+0000", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0237000000000001e-05 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.580661+0000", 2024-09-15T21:00:40.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5488999999999999e-05 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.682572+0000", 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101910639 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.682597+0000", 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5182e-05 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.683967+0000", 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013699059999999999 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.684039+0000", 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1889000000000002e-05 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.070 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s7 37/28 ECSubRead(tid=28, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:40.071 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.360528046000001, 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20065597199999999, 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676186+0000", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7430000000000001e-06 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676208+0000", 2024-09-15T21:00:40.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1985e-05 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676247+0000", 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8852e-05 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.776372+0000", 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10012509 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.876689+0000", 2024-09-15T21:00:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100317034 2024-09-15T21:00:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.876841+0000", 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151268 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=76, reqid=client.4641.0:244, at_version=39'36, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9332828370000001, 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20322754300000001, 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103431+0000", 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4700000000000001e-07 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103932+0000", 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000501845 2024-09-15T21:00:40.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103966+0000", 2024-09-15T21:00:40.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3621999999999998e-05 2024-09-15T21:00:40.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103990+0000", 2024-09-15T21:00:40.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3739000000000001e-05 2024-09-15T21:00:40.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.204334+0000", 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100344055 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.204352+0000", 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8236e-05 2024-09-15T21:00:40.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.306570+0000", 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10221767 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.306657+0000", 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7628999999999997e-05 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=77, reqid=osd.5.0:77, at_version=39'38, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.9292555070000006, 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20007127199999999, 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107459+0000", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2790000000000001e-06 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107499+0000", 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9437999999999997e-05 2024-09-15T21:00:40.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107531+0000", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1862000000000001e-05 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107554+0000", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3569000000000001e-05 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.305963+0000", 2024-09-15T21:00:40.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19840924300000001 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.305979+0000", 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5248999999999999e-05 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.307408+0000", 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001428868 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.307528+0000", 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120764 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 39/28 ECSubWrite(tid=32, reqid=client.4641.0:255, at_version=39'10, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.227299+0000", 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8094139879999993, 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102394104, 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227299+0000", 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227299+0000", 2024-09-15T21:00:40.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227300+0000", 2024-09-15T21:00:40.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6959999999999999e-06 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227340+0000", 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9545000000000002e-05 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227375+0000", 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4931000000000001e-05 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.227398+0000", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2821999999999999e-05 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.327689+0000", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10029115500000001 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.327713+0000", 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4666000000000001e-05 2024-09-15T21:00:40.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.329593+0000", 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018794739999999999 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.329693+0000", 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9815000000000006e-05 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 39/28 ECSubWrite(tid=33, reqid=client.4641.0:256, at_version=39'11, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8085924260000006, 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104120988, 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228122+0000", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228164+0000", 2024-09-15T21:00:40.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2011000000000001e-05 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228204+0000", 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9780999999999997e-05 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228227+0000", 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3405999999999999e-05 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.328399+0000", 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100171435 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.328423+0000", 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4485e-05 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.332178+0000", 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037545349999999998 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.332241+0000", 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3275999999999995e-05 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=78, reqid=client.4641.0:257, at_version=39'40, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7238434439999999, 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20214031900000001, 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312871+0000", 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312934+0000", 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2856999999999998e-05 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312974+0000", 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9790999999999999e-05 2024-09-15T21:00:40.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413116+0000", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014285000000001 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513330+0000", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100213416 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513346+0000", 2024-09-15T21:00:40.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5852e-05 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514911+0000", 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001565108 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.515009+0000", 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8654999999999996e-05 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s4 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6634495450000006, 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.201922037, 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373265+0000", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.199e-06 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373383+0000", 2024-09-15T21:00:40.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117367 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373415+0000", 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1931999999999999e-05 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373430+0000", 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5894999999999999e-05 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473678+0000", 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100247579 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473714+0000", 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5858000000000001e-05 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575119+0000", 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101405157 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575185+0000", 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6050000000000006e-05 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s4 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6595651280000006, 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19810077700000001, 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.107 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377149+0000", 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.787e-06 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377181+0000", 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1494e-05 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377211+0000", 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0015e-05 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377226+0000", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5282999999999999e-05 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.474792+0000", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097565747999999994 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.474821+0000", 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9697999999999999e-05 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575194+0000", 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100372138 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575248+0000", 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4614000000000003e-05 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 39/28 ECSubWrite(tid=79, reqid=osd.5.0:79, at_version=39'14, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6235426089999994, 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16219983700000001, 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413171+0000", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3170000000000001e-06 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413238+0000", 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6926000000000003e-05 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413269+0000", 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0491e-05 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413292+0000", 2024-09-15T21:00:40.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2812000000000001e-05 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513803+0000", 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10051183 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513828+0000", 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5116000000000001e-05 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575263+0000", 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061434158000000003 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575370+0000", 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107187 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 39/28 ECSubWrite(tid=80, reqid=client.4641.0:250, at_version=39'15, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.6233888810000003, 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16208093700000001, 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413325+0000", 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413482+0000", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157262 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413501+0000", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9398999999999998e-05 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513640+0000", 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100138827 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514646+0000", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010056279999999999 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514676+0000", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9808e-05 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575374+0000", 2024-09-15T21:00:40.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060698479 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575405+0000", 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0550999999999999e-05 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 39/28 ECSubWrite(tid=34, reqid=client.4641.0:256, at_version=0'0, trim_to=0'0, roll_forward_to=39'11))", 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:40.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4059749250000007, 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12595379500000001, 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630740+0000", 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8369999999999999e-06 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630759+0000", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9888999999999999e-05 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630790+0000", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630816+0000", 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5664999999999999e-05 2024-09-15T21:00:40.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.731158+0000", 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100342483 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.731174+0000", 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5463000000000001e-05 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.756611+0000", 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025437517999999999 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.756691+0000", 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0072000000000002e-05 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.es7 39/28 ECSubWrite(tid=36, reqid=client.4641.0:271, at_version=39'7, trim_to=0'0, roll_forward_to=33'5))", 2024-09-15T21:00:40.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.654949+0000", 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3817637319999996, 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102573023, 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.654949+0000", 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.654949+0000", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.654951+0000", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1390000000000002e-06 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.655006+0000", 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5464000000000003e-05 2024-09-15T21:00:40.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.655048+0000", 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1244e-05 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.655071+0000", 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3572000000000001e-05 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.755464+0000", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10039263700000001 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.755494+0000", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0295e-05 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.757453+0000", 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019588129999999998 2024-09-15T21:00:40.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.757522+0000", 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8858999999999999e-05 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.cs3 39/28 ECSubRead(tid=87, to_read={3:3f0b4dcc:::smithi02732947-9:head=0,4096,0}, subchunks={3:3f0b4dcc:::smithi02732947-9:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:40.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.1583998520000005, 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.158771838, 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878315+0000", 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7829999999999999e-06 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878334+0000", 2024-09-15T21:00:40.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9043e-05 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878384+0000", 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0587000000000002e-05 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.978506+0000", 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100122275 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.036912+0000", 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058405743000000003 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037085+0000", 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017240699999999999 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s4 39/28 ECSubWrite(tid=42, reqid=client.4641.0:265, at_version=39'30, trim_to=0'0, roll_forward_to=39'28))", 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.934922+0000", 2024-09-15T21:00:40.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.1017910690000008, 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10254123499999999, 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934922+0000", 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934922+0000", 2024-09-15T21:00:40.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934923+0000", 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.629e-06 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934960+0000", 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7242e-05 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934987+0000", 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6917999999999999e-05 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.935002+0000", 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4466e-05 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.035275+0000", 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10027320200000001 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.035305+0000", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9989e-05 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037387+0000", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020819800000000002 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037463+0000", 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5809000000000006e-05 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.140 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:40.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:40.142 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.137+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:40.142 INFO:tasks.rados.rados.0.smithi027.stdout:129: done (2 left) 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:130: read oid 25 snap -1 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:130: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:131: read oid 44 snap -1 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:131: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:132: rollback oid 8 current snap is 8 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 8 to 6 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:127: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-15T21:00:40.143 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:40.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:00:40.180 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 569193 2024-09-15T21:00:40.194 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 556579 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: finishing write tid 5 to smithi02732947-31 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: oid 31 updating version 0 to 33 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: oid 31 updating version 33 to 36 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: oid 31 version 36 is already newer than 34 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: oid 31 version 36 is already newer than 35 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 31 v 36 (ObjNum 73 snap 8 seq_num 73) dirty exists 2024-09-15T21:00:40.200 INFO:tasks.rados.rados.0.smithi027.stdout:124: left oid 31 (ObjNum 73 snap 8 seq_num 73) 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:124: done (4 left) 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:127: done (3 left) 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:133: read oid 18 snap -1 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:133: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 21 current snap is 8 2024-09-15T21:00:40.201 INFO:tasks.rados.rados.0.smithi027.stdout:134: seq_num 76 ranges {344064=507904,851968=507904,1359872=262144} 2024-09-15T21:00:40.209 INFO:tasks.rados.rados.0.smithi027.stdout:134: writing smithi02732947-21 from 344064 to 851968 tid 1 2024-09-15T21:00:40.210 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.205+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:40.210 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.209+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:00:40.210 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.209+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:40.212 INFO:tasks.rados.rados.0.smithi027.stdout:134: writing smithi02732947-21 from 851968 to 1359872 tid 2 2024-09-15T21:00:40.214 INFO:tasks.rados.rados.0.smithi027.stdout:134: writing smithi02732947-21 from 1359872 to 1622016 tid 3 2024-09-15T21:00:40.214 INFO:tasks.rados.rados.0.smithi027.stdout:135: rmattr oid 34 current snap is 8 2024-09-15T21:00:40.214 INFO:tasks.rados.rados.0.smithi027.stdout:135: done (5 left) 2024-09-15T21:00:40.214 INFO:tasks.rados.rados.0.smithi027.stdout:136: snap_remove snap 7 2024-09-15T21:00:40.242 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:40.241+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:40.265 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.261+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:00:40.265 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.261+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:00:40.318 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:40.318 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.bs4 48/28 ECSubRead(tid=114, to_read={3:d1127d8d:::smithi02732947-18:head=0,352256,0}, subchunks={3:d1127d8d:::smithi02732947-18:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.301872+0000", 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.016709755999999999, 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016754287999999999, 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301872+0000", 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301872+0000", 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301873+0000", 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4000000000000001e-07 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301883+0000", 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5259999999999999e-06 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301906+0000", 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3436000000000001e-05 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301934+0000", 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8255e-05 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.322 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/28 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=48'38, trim_to=0'0, roll_forward_to=47'36))", 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.301970+0000", 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.016612121000000001, 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016735085, 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301970+0000", 2024-09-15T21:00:40.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301970+0000", 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.301971+0000", 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302049+0000", 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7826999999999999e-05 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302073+0000", 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4113000000000001e-05 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302100+0000", 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7795000000000001e-05 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/28 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=48'40, trim_to=0'0, roll_forward_to=47'36))", 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.302128+0000", 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.016454267000000002, 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.016642239, 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302128+0000", 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302128+0000", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302129+0000", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6599999999999995e-07 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302172+0000", 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3498999999999998e-05 2024-09-15T21:00:40.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302197+0000", 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4405000000000001e-05 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.302214+0000", 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7356000000000001e-05 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:00:40.328 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:00:40.329 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:40.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:40.343 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:40.339+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:40.367 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:40.365+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:00:40.421 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:40.421 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:40.421 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.37570061876590854 2024-09-15T21:00:40.421 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.37570061876590854 2024-09-15T21:00:40.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:40.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:00:40.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6 to read 0,1,4,5 2024-09-15T21:00:40.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.451+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:40.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.451+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:00:40.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,4,5 2024-09-15T21:00:40.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:40.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:40.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:00:40.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:40.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.459+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:40.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.459+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:40.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.459+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:40.466 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:40.465+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:40.500 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:40.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:40.501 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:40.501 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:40.501 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:40.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:40.546 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.541+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s1 33/28 ECSubWrite(tid=9, reqid=client.4641.0:134, at_version=33'6, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.299138+0000", 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.339344347000001, 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202239318, 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299138+0000", 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299138+0000", 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299140+0000", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6360000000000001e-06 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299180+0000", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0510999999999999e-05 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299221+0000", 2024-09-15T21:00:40.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0587999999999998e-05 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299245+0000", 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3884e-05 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.499635+0000", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20039016300000001 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.499658+0000", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3495000000000001e-05 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501312+0000", 2024-09-15T21:00:40.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016533609999999999 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501377+0000", 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5679999999999995e-05 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.055617808000001, 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:00:40.646 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:00:40.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:00:40.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.054910451000001, 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:00:40.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:00:40.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.054337308000001, 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:00:40.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:00:40.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.045488094, 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.294194122, 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592996+0000", 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.73e-06 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593005+0000", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1689999999999998e-06 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593035+0000", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9975e-05 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.693128+0000", 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100092589 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887125+0000", 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19399696999999999 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887188+0000", 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3689000000000006e-05 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.954095367000001, 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:00:40.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=17, last_complete=33'3, committed=1, applied=1))", 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.946269990999999, 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295470228, 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692214+0000", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6729999999999999e-06 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692223+0000", 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8899999999999996e-06 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692249+0000", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6310999999999999e-05 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792473+0000", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100224217 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987650+0000", 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19517695600000001 2024-09-15T21:00:40.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987683+0000", 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2181000000000002e-05 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.847837067, 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:00:40.678 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:00:40.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:00:40.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.847724632, 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:40.684 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:00:40.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:00:40.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=19, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.840290323, 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29016010199999998, 2024-09-15T21:00:40.690 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798193+0000", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.443e-06 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798202+0000", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1759999999999993e-06 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798225+0000", 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3342000000000001e-05 2024-09-15T21:00:40.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798237+0000", 2024-09-15T21:00:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2466000000000001e-05 2024-09-15T21:00:40.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088293+0000", 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29005556199999999 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088352+0000", 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9113000000000001e-05 2024-09-15T21:00:40.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=17, last_complete=33'3, committed=1, applied=1))", 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.833860102999999, 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28387544599999998, 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804624+0000", 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804635+0000", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0769e-05 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804663+0000", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7622000000000001e-05 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804682+0000", 2024-09-15T21:00:40.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9595000000000001e-05 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088463+0000", 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28378098899999998 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088498+0000", 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4461000000000001e-05 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:00:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:00:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.833767262, 2024-09-15T21:00:40.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38400288500000002, 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:00:40.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804716+0000", 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804722+0000", 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0689999999999998e-06 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804740+0000", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7978999999999998e-05 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904872+0000", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100132263 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188682+0000", 2024-09-15T21:00:40.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28381031600000001 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188718+0000", 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=19, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:40.704 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.904948+0000", 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.733534253, 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28383507099999999, 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904948+0000", 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904948+0000", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904949+0000", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3540000000000001e-06 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904958+0000", 2024-09-15T21:00:40.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1659999999999994e-06 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904981+0000", 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3808e-05 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.905000+0000", 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8702000000000002e-05 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188759+0000", 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28375922599999998 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188783+0000", 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3815000000000002e-05 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:00:40.708 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.198035263, 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29972135599999999, 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:00:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440448+0000", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.003e-06 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440453+0000", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1660000000000002e-06 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440474+0000", 2024-09-15T21:00:40.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0801999999999998e-05 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440487+0000", 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3339000000000001e-05 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740144+0000", 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29965620500000001 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740168+0000", 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4841000000000002e-05 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.197975627, 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29977237099999998, 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:00:40.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440512+0000", 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5630000000000002e-06 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440528+0000", 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6049000000000002e-05 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440540+0000", 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1846e-05 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740241+0000", 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29970132900000002 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740279+0000", 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7931e-05 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.197926444, 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299787898, 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:00:40.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7700000000000004e-07 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440561+0000", 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440577+0000", 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6365999999999999e-05 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440591+0000", 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3985000000000001e-05 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740326+0000", 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29973503400000001 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740344+0000", 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7756000000000001e-05 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:00:40.719 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.197689842000001, 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29959435299999998, 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440794+0000", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440800+0000", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9869999999999996e-06 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440821+0000", 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1308e-05 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440856+0000", 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740372+0000", 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29951594300000001 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740387+0000", 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4537e-05 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:00:40.723 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.197601393999999, 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39968706100000001, 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440882+0000", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440886+0000", 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440911+0000", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494999999999998e-05 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440924+0000", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3142999999999999e-05 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840527+0000", 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39960277900000002 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840568+0000", 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.196877873, 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39904099900000001, 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441605+0000", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441611+0000", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441652+0000", 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0890999999999999e-05 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441671+0000", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9059000000000002e-05 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840619+0000", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39894794099999997 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840645+0000", 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6064999999999999e-05 2024-09-15T21:00:40.730 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s6 38/28 ECSubWrite(tid=71, reqid=osd.5.0:71, at_version=38'30, trim_to=0'0, roll_forward_to=37'28))", 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:30.079617+0000", 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.558865363000001, 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20624039499999999, 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079617+0000", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079617+0000", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079618+0000", 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.133e-06 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079656+0000", 2024-09-15T21:00:40.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8294999999999998e-05 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079679+0000", 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2521999999999999e-05 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.179775+0000", 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10009625900000001 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.279967+0000", 2024-09-15T21:00:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10019188399999999 2024-09-15T21:00:40.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.279981+0000", 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4486e-05 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.285760+0000", 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057784109999999998 2024-09-15T21:00:40.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.285857+0000", 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7405000000000007e-05 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:40.740 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:40.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:40.742 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:40.735+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:40.842 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:40.839+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:40.862 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.861+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:40.862 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.861+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:00:40.863 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.861+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:00:40.863 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:40.861+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:00:40.865 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:40.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:40.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.865 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 48/28 ECSubReadReply(tid=46, attrs_read=0))", 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.863145+0000", 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0014794700000000001, 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0015274170000000001, 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.863145+0000", 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.863145+0000", 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.863145+0000", 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.863145+0000", 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.863188+0000", 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3424000000000003e-05 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 48/28 ECSubWriteReply(tid=45, last_complete=48'18, committed=1, applied=1))", 2024-09-15T21:00:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.864234+0000", 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.00039052800000000001, 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00056463599999999998, 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864234+0000", 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864234+0000", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864235+0000", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864245+0000", 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9890000000000004e-06 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864272+0000", 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6514999999999999e-05 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864286+0000", 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3566e-05 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.45709171300000001, 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.45740824200000002, 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:00:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:00:40.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 48/28 ECSubReadReply(tid=46, attrs_read=0))", 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.860564+0000", 2024-09-15T21:00:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0040602839999999999, 2024-09-15T21:00:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0045934590000000003, 2024-09-15T21:00:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:40.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:40.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.860564+0000", 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.860564+0000", 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.860566+0000", 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5000000000000002e-06 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.860589+0000", 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2682000000000001e-05 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.860615+0000", 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6318000000000001e-05 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:40.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 48/28 ECSubReadReply(tid=46, attrs_read=0))", 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.862804+0000", 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.001820405, 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024647890000000002, 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:40.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862804+0000", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862804+0000", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862805+0000", 2024-09-15T21:00:40.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7400000000000003e-07 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862819+0000", 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4775e-05 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862832+0000", 2024-09-15T21:00:40.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2646000000000001e-05 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.862842+0000", 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.027e-05 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.864453+0000", 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001610361 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.885 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:40.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:40.886 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:40.886 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 5 2024-09-15T21:00:40.886 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:40.887 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.863+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:40.887 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.863+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:40.887 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.863+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:00:40.888 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:40.947 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:40.945+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:40.969 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.963+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,6 to read 0,1,4,5 2024-09-15T21:00:40.969 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.963+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3 to read 0,1,4,5 2024-09-15T21:00:40.971 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.967+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:00:40.971 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:40.967+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:00:41.024 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:41.025 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:41.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:41.049 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:41.045+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:41.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:41.067+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:41.140 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:00:41.140 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.1 to 0.375701 (602d) 2024-09-15T21:00:41.146 INFO:tasks.rados.rados.0.smithi027.stdout:136: done (5 left) 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:137: rmattr oid 12 current snap is 8 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:137: done (5 left) 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:138: snap_create 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:132: finishing rollback tid 0 to smithi02732947-8 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:132: finishing rollback tid 1 to smithi02732947-8 2024-09-15T21:00:41.147 INFO:tasks.rados.rados.0.smithi027.stdout:132: finishing rollback tid 2 to smithi02732947-8 2024-09-15T21:00:41.148 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 8 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T21:00:41.148 INFO:tasks.rados.rados.0.smithi027.stdout:134: finishing write tid 1 to smithi02732947-21 2024-09-15T21:00:41.148 INFO:tasks.rados.rados.0.smithi027.stdout:134: finishing write tid 2 to smithi02732947-21 2024-09-15T21:00:41.148 INFO:tasks.rados.rados.0.smithi027.stdout:134: finishing write tid 3 to smithi02732947-21 2024-09-15T21:00:41.148 INFO:tasks.rados.rados.0.smithi027.stdout:134: finishing write tid 4 to smithi02732947-21 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: finishing write tid 5 to smithi02732947-21 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: oid 21 updating version 0 to 15 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: oid 21 updating version 15 to 18 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: oid 21 version 18 is already newer than 16 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: oid 21 version 18 is already newer than 17 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 21 v 18 (ObjNum 76 snap 8 seq_num 76) dirty exists 2024-09-15T21:00:41.149 INFO:tasks.rados.rados.0.smithi027.stdout:134: left oid 21 (ObjNum 76 snap 8 seq_num 76) 2024-09-15T21:00:41.150 INFO:tasks.rados.rados.0.smithi027.stdout:130: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T21:00:41.153 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:41.148+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:41.162 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 174920 2024-09-15T21:00:41.170 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 969555 2024-09-15T21:00:41.172 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:41.169+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:41.173 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:41.169+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:41.173 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:41.189 INFO:tasks.rados.rados.0.smithi027.stdout:131: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:41.190 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:41.197 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:41.204 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:41.227 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:41.227 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:41.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.960975862000002, 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.230 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.233 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.960677723, 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:00:41.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.960653429000001, 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.239 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:00:41.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:41.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.243 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.753590967000001, 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:00:41.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.246 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.247 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.es4 32/28 ECSubWrite(tid=13, reqid=client.4641.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.678212753, 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32195022499999998, 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:41.248 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547350+0000", 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.249 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547377+0000", 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6630000000000001e-05 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547411+0000", 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3708999999999999e-05 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547429+0000", 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867891+0000", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32046238700000002 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867902+0000", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1385e-05 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869267+0000", 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013649020000000001 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869298+0000", 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0997000000000002e-05 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.es4 32/28 ECSubRead(tid=14, to_read={3:770058af:::smithi02732947-19:head=0,4096,0}, subchunks={3:770058af:::smithi02732947-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.675820601000002, 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31956152100000002, 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.253 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549742+0000", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.516e-06 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549757+0000", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5248999999999999e-05 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549789+0000", 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.205e-05 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549807+0000", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7870999999999999e-05 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868926+0000", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31911888799999999 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869302+0000", 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037594700000000001 2024-09-15T21:00:41.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=14, attrs_read=0))", 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.667111139999999, 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30910945000000001, 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558450+0000", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558451+0000", 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4130000000000001e-06 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558472+0000", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1038000000000001e-05 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558497+0000", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4620000000000001e-05 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558512+0000", 2024-09-15T21:00:41.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4902000000000001e-05 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867529+0000", 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30901767499999999 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867559+0000", 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9802e-05 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.666670466999999, 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30970041700000001, 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:41.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558890+0000", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558892+0000", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5e-06 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558915+0000", 2024-09-15T21:00:41.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3527e-05 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558944+0000", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8827e-05 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.558962+0000", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7320999999999999e-05 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868104+0000", 2024-09-15T21:00:41.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30914279099999997 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868591+0000", 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048645099999999999 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.666065229000001, 2024-09-15T21:00:41.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30937452599999998, 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559496+0000", 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559501+0000", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.339e-06 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559517+0000", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6084999999999999e-05 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559537+0000", 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9911e-05 2024-09-15T21:00:41.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.559552+0000", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5424999999999999e-05 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868839+0000", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30928677999999998 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868870+0000", 2024-09-15T21:00:41.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0985999999999998e-05 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.100354923999999, 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.267 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:00:41.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:00:41.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:41.272 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.000189864999999, 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:41.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:00:41.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.000145195, 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:00:41.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:00:41.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:00:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:353 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.082807217, 2024-09-15T21:00:41.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31468109900000002, 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 353 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142757+0000", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3560000000000001e-06 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142771+0000", 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3793e-05 2024-09-15T21:00:41.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142807+0000", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6093999999999997e-05 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142833+0000", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6475e-05 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143017+0000", 2024-09-15T21:00:41.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018384099999999999 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143040+0000", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671999999999999e-05 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.147993+0000", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049528979999999998 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.148147+0000", 2024-09-15T21:00:41.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.150417+0000", 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022698869999999999 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157031+0000", 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0066142179999999998 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.457335+0000", 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30030378299999999 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.457435+0000", 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9841999999999997e-05 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:00:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.082673596, 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:00:41.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.296 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.0825822089999999, 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:00:41.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.302 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:359 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.91923082, 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35050568300000001, 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 359 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306332+0000", 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8339999999999999e-06 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306628+0000", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029618500000000002 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306645+0000", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6518e-05 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306653+0000", 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1960000000000008e-06 2024-09-15T21:00:41.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306721+0000", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7945000000000001e-05 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306833+0000", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112538 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.308213+0000", 2024-09-15T21:00:41.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001379915 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.309315+0000", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001102107 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656634+0000", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34731901900000001 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656836+0000", 2024-09-15T21:00:41.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020142599999999999 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.91888504900000001, 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.308 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:00:41.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:00:41.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.313 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:361 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~262144] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.81844001899999996, 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35217888200000003, 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 361 2024-09-15T21:00:41.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407122+0000", 2024-09-15T21:00:41.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.567e-06 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407417+0000", 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029430100000000001 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407438+0000", 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1401000000000001e-05 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407450+0000", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2036e-05 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409764+0000", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023141680000000001 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409913+0000", 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000148777 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.411230+0000", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00131726 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413635+0000", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024047719999999999 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759187+0000", 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34555227100000002 2024-09-15T21:00:41.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759300+0000", 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112329 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:362 3.6s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (56) in=7b] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.81808908199999997, 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35221260100000001, 2024-09-15T21:00:41.319 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 362 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407473+0000", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407479+0000", 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5899999999999996e-06 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407492+0000", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2435999999999999e-05 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407501+0000", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5589999999999994e-06 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412116+0000", 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046143970000000001 2024-09-15T21:00:41.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412250+0000", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013454100000000001 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413018+0000", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076778700000000003 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413954+0000", 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000935691 2024-09-15T21:00:41.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759576+0000", 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34562273999999998 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759684+0000", 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010788 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.81802828599999999, 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:41.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:00:41.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:00:41.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:00:41.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.329 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:41.330 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:41.330 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:41.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:41.332 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:41.273+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:41.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:41.327+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:41.432 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:41.429+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:41.509 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:41.509 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:41.509 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:41.509 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:41.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:41.533 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:41.532+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:41.633 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:41.627+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:41.676 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:41.695 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:41.695 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:41.695 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:41.695 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:41.695 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:41.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:41.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:41.727+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:41.820 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:41.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:41.831+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:41.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:41.837 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:41.854 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:41.854 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:41.871 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:41.871 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:41.880 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:41.880 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:41.891 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:41.892 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:41.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:41.913 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:41.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:41.935 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:41.936 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:41.933+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:41.956 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:41.956 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:41.979 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:41.979 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:42.001 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:42.001 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:42.011 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.012 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.024 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:42.025 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:42.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:42.048 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:42.049 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:42.045+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:42.137 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:42.132+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:42.154 INFO:tasks.rados.rados.0.smithi027.stdout:130: done (5 left) 2024-09-15T21:00:42.154 INFO:tasks.rados.rados.0.smithi027.stdout:131: done (4 left) 2024-09-15T21:00:42.154 INFO:tasks.rados.rados.0.smithi027.stdout:132: done (3 left) 2024-09-15T21:00:42.154 INFO:tasks.rados.rados.0.smithi027.stdout:134: done (2 left) 2024-09-15T21:00:42.155 INFO:tasks.rados.rados.0.smithi027.stdout:138: done (1 left) 2024-09-15T21:00:42.155 INFO:tasks.rados.rados.0.smithi027.stdout:139: copy_from oid 33 from oid 36 current snap is 9 2024-09-15T21:00:42.155 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 42 current snap is 9 2024-09-15T21:00:42.155 INFO:tasks.rados.rados.0.smithi027.stdout:140: seq_num 77 ranges {229376=524288,753664=524288,1277952=245760} 2024-09-15T21:00:42.168 INFO:tasks.rados.rados.0.smithi027.stdout:140: writing smithi02732947-42 from 229376 to 753664 tid 1 2024-09-15T21:00:42.177 INFO:tasks.rados.rados.0.smithi027.stdout:140: writing smithi02732947-42 from 753664 to 1277952 tid 2 2024-09-15T21:00:42.181 INFO:tasks.rados.rados.0.smithi027.stdout:140: writing smithi02732947-42 from 1277952 to 1523712 tid 3 2024-09-15T21:00:42.181 INFO:tasks.rados.rados.0.smithi027.stdout:141: setattr oid 37 current snap is 9 2024-09-15T21:00:42.186 INFO:tasks.rados.rados.0.smithi027.stdout:142: snap_remove snap 8 2024-09-15T21:00:42.188 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.189 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:42.238 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:42.236+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.328 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.338 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:42.335+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:42.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:42.359 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:42.357+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:42.440 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:42.437+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:42.462 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.462 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:42.566 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:42.565+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:42.642 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:42.641+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:42.643 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.643 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:42.743 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:42.739+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:42.827 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.827 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:42.843 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:42.840+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:42.945 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:42.939+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:42.978 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:42.978 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:42.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:43.083 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:43.079+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:43.147 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.145+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:43.154 INFO:tasks.rados.rados.0.smithi027.stdout:142: done (4 left) 2024-09-15T21:00:43.154 INFO:tasks.rados.rados.0.smithi027.stdout:143: rmattr oid 39 current snap is 9 2024-09-15T21:00:43.154 INFO:tasks.rados.rados.0.smithi027.stdout:143: done (4 left) 2024-09-15T21:00:43.154 INFO:tasks.rados.rados.0.smithi027.stdout:144: copy_from oid 32 from oid 48 current snap is 9 2024-09-15T21:00:43.155 INFO:tasks.rados.rados.0.smithi027.stdout:145: snap_remove snap 6 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.157 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.153+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.162 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.157+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.163 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.163 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.163 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.164 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.164 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.164 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.166 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.166 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.166 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.161+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:00:43.178 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:43.178 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:43.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:43.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:43.279+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:43.348 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:43.345+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:43.362 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.356+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:43.362 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.356+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:00:43.362 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.356+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,4,5 2024-09-15T21:00:43.366 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:43.366 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:43.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:43.450 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.448+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:43.462 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.460+0000 7f84b8505640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,4,5 2024-09-15T21:00:43.530 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:00:43.535 INFO:tasks.rados.rados.0.smithi027.stdout:145: done (5 left) 2024-09-15T21:00:43.535 INFO:tasks.rados.rados.0.smithi027.stdout:146: read oid 44 snap -1 2024-09-15T21:00:43.535 INFO:tasks.rados.rados.0.smithi027.stdout:146: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:43.535 INFO:tasks.rados.rados.0.smithi027.stdout:147: setattr oid 50 current snap is 9 2024-09-15T21:00:43.535 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 16 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T21:00:43.538 INFO:tasks.rados.rados.0.smithi027.stdout:141: done (6 left) 2024-09-15T21:00:43.538 INFO:tasks.rados.rados.0.smithi027.stdout:148: copy_from oid 26 from oid 47 current snap is 9 2024-09-15T21:00:43.538 INFO:tasks.rados.rados.0.smithi027.stdout:149: copy_from oid 15 from oid 14 current snap is 9 2024-09-15T21:00:43.538 INFO:tasks.rados.rados.0.smithi027.stdout:150: snap_create 2024-09-15T21:00:43.550 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.545+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:43.551 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.545+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:43.551 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.549+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:00:43.551 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:43.547+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:43.563 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:43.563 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:43.571 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.569+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.571 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.569+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:43.575 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:43.573+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:00:43.581 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:43.651 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:43.648+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:43.671 INFO:tasks.rados.rados.0.smithi027.stdout:144: finishing copy_from to smithi02732947-32 2024-09-15T21:00:43.671 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 32 v 5 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-15T21:00:43.672 INFO:tasks.rados.rados.0.smithi027.stdout:144: finishing copy_from racing read to smithi02732947-32 2024-09-15T21:00:43.753 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:43.748+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:43.754 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:43.755 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:43.755 INFO:tasks.rados.rados.0.smithi027.stdout:149: finishing copy_from racing read to smithi02732947-15 2024-09-15T21:00:43.771 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:43.786 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.784+0000 7f84b9507640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:43.787 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.784+0000 7f84bd50f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:00:43.787 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:43.784+0000 7f84bd50f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:00:43.854 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:43.849+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:43.858 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:43.856+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:43.858 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:43.856+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:43.858 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:43.856+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:00:43.864 INFO:tasks.rados.rados.0.smithi027.stdout:139: finishing copy_from racing read to smithi02732947-33 2024-09-15T21:00:43.897 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:43.897 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:43.913 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:43.958 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:43.952+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,7 to read 0,1,4,5 2024-09-15T21:00:43.958 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:43.955+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:44.040 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:44.040 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:44.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:44.056 INFO:tasks.rados.rados.0.smithi027.stdout:148: finishing copy_from racing read to smithi02732947-26 2024-09-15T21:00:44.057 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:44.053+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:44.062 INFO:tasks.rados.rados.0.smithi027.stdout:140: finishing write tid 1 to smithi02732947-42 2024-09-15T21:00:44.065 INFO:tasks.rados.rados.0.smithi027.stdout:139: finishing copy_from to smithi02732947-33 2024-09-15T21:00:44.066 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 33 v 12 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T21:00:44.070 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:44.084 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:44.081+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:44.085 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:44.081+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,4,5 2024-09-15T21:00:44.158 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:44.151+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:44.185 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:44.181+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4 to read 0,1,4,5 2024-09-15T21:00:44.228 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:44.228 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:44.233 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:44.234 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:44.242 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:44.257 INFO:tasks.rados.rados.0.smithi027.stdout:133: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:00:44.260 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 570917 2024-09-15T21:00:44.268 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:44.269 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:44.269 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:44.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.275+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:44.280 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:44.275+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:44.284 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 200181 2024-09-15T21:00:44.287 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:44.297 INFO:tasks.rados.rados.0.smithi027.stdout:148: finishing copy_from to smithi02732947-26 2024-09-15T21:00:44.297 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 26 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:00:44.297 INFO:tasks.rados.rados.0.smithi027.stdout:149: finishing copy_from to smithi02732947-15 2024-09-15T21:00:44.297 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 15 v 10 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T21:00:44.307 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:44.307 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:44.359 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:44.355+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:44.381 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.375+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,4,5 2024-09-15T21:00:44.381 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.375+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4 to read 0,1,4,5 2024-09-15T21:00:44.411 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:44.411 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:44.426 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:44.461 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:44.457+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:44.481 INFO:tasks.rados.rados.0.smithi027.stdout:140: finishing write tid 2 to smithi02732947-42 2024-09-15T21:00:44.538 INFO:tasks.rados.rados.0.smithi027.stdout:133: done (8 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:139: done (7 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:144: done (6 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:148: done (5 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:149: done (4 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:150: done (3 left) 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 4 current snap is 10 2024-09-15T21:00:44.539 INFO:tasks.rados.rados.0.smithi027.stdout:151: seq_num 78 ranges {983040=557056,1540096=671744,2211840=98304} 2024-09-15T21:00:44.561 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:44.555+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:44.565 INFO:tasks.rados.rados.0.smithi027.stdout:151: writing smithi02732947-4 from 983040 to 1540096 tid 1 2024-09-15T21:00:44.576 INFO:tasks.rados.rados.0.smithi027.stdout:151: writing smithi02732947-4 from 1540096 to 2211840 tid 2 2024-09-15T21:00:44.577 INFO:tasks.rados.rados.0.smithi027.stdout:151: writing smithi02732947-4 from 2211840 to 2310144 tid 3 2024-09-15T21:00:44.578 INFO:tasks.rados.rados.0.smithi027.stdout:152: snap_create 2024-09-15T21:00:44.583 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:44.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.579+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:44.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.579+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:44.600 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:44.600 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:44.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:44.604 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:44.616 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:44.632 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:44.632 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:44.663 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:44.659+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:44.686 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.679+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:00:44.687 INFO:tasks.rados.rados.0.smithi027.stdout:140: finishing write tid 3 to smithi02732947-42 2024-09-15T21:00:44.687 INFO:tasks.rados.rados.0.smithi027.stdout:140: finishing write tid 4 to smithi02732947-42 2024-09-15T21:00:44.689 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.683+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:44.763 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:44.760+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:44.792 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:44.792 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:44.792 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:44.792 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:44.792 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:44.797 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:44.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:44.832 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:44.832 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:44.865 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:44.861+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:44.865 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:44.865 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:44.899 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:44.966 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:44.959+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:44.986 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.524298215999998, 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33373998100000002, 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:44.987 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457901+0000", 2024-09-15T21:00:44.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457904+0000", 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457918+0000", 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4093999999999999e-05 2024-09-15T21:00:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457944+0000", 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5986000000000002e-05 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.457965+0000", 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0954e-05 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:44.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458117+0000", 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015210500000000001 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.458129+0000", 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.149e-05 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539011+0000", 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.080882495999999998 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for peered", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.539035+0000", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3439999999999999e-05 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788238+0000", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.24920330900000001 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.788633+0000", 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039541700000000001 2024-09-15T21:00:44.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.789053+0000", 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00041907000000000001 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791200+0000", 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021477969999999999 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:44.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791447+0000", 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024629799999999999 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791572+0000", 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012488899999999999 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:57.791641+0000", 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9843000000000002e-05 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:44.994 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.363949351000002, 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:44.995 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:44.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:00:44.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:00:45.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.750655092000002, 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:00:45.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:00:45.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.750470389999997, 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.007 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:00:45.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:00:45.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:00:45.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.545194379000002, 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:00:45.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:00:45.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.018 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.544869218000002, 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:00:45.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:00:45.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.544828361, 2024-09-15T21:00:45.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:45.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:00:45.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.544779615000003, 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.035 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:00:45.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.544464703999999, 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:00:45.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:00:45.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 44.544386637999999, 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:45.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:45.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:00:45.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:00:45.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 42.849928321, 2024-09-15T21:00:45.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31281666299999999, 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 71 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132271+0000", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132272+0000", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132278+0000", 2024-09-15T21:00:45.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7570000000000001e-06 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132291+0000", 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2703e-05 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132299+0000", 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0069999999999997e-06 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139452+0000", 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0071530079999999998 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139562+0000", 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000110432 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.139743+0000", 2024-09-15T21:00:45.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018057899999999999 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142717+0000", 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002974661 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444884+0000", 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216663399999999 2024-09-15T21:00:45.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.444969+0000", 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4452999999999999e-05 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445088+0000", 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119449 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 42.849887688000003, 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31312449999999997, 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.059 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 72 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132312+0000", 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132313+0000", 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132422+0000", 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109033 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132439+0000", 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7586000000000001e-05 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132449+0000", 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6199999999999994e-06 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140318+0000", 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0078687559999999993 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140430+0000", 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112589 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.140615+0000", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018495600000000001 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142725+0000", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021100300000000002 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445273+0000", 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30254736500000001 2024-09-15T21:00:45.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445366+0000", 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3249000000000001e-05 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445436+0000", 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0406000000000003e-05 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.065 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.066 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.066 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:45.066 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 42.849732064999998, 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31325695799999997, 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 73 2024-09-15T21:00:45.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132468+0000", 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132469+0000", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132475+0000", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4999999999999996e-06 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132493+0000", 2024-09-15T21:00:45.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7669999999999999e-05 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132507+0000", 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3966e-05 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141793+0000", 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0092860819999999993 2024-09-15T21:00:45.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.141919+0000", 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000126396 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.142125+0000", 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020598799999999999 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.143405+0000", 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012801570000000001 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445573+0000", 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30216763699999999 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445651+0000", 2024-09-15T21:00:45.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8239999999999996e-05 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445725+0000", 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3329000000000002e-05 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 42.849647279999999, 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31343764099999999, 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.074 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 74 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132552+0000", 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132554+0000", 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.327e-06 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132565+0000", 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0967e-05 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.132584+0000", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.984e-05 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232738+0000", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10015367 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.232890+0000", 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015175200000000001 2024-09-15T21:00:45.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233152+0000", 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026189099999999999 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233532+0000", 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038017699999999999 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239440+0000", 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.005908122 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445839+0000", 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20639856300000001 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445917+0000", 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8055999999999999e-05 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.445990+0000", 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3275999999999994e-05 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.080 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.081 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.081 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.081 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 42.749156769000002, 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31253853100000001, 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.082 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 76 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233043+0000", 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233045+0000", 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7370000000000001e-06 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233061+0000", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6453e-05 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233105+0000", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4063000000000001e-05 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.233123+0000", 2024-09-15T21:00:45.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7937000000000001e-05 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239508+0000", 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0063845940000000004 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239647+0000", 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013887599999999999 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.239834+0000", 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018760200000000001 2024-09-15T21:00:45.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.243229+0000", 2024-09-15T21:00:45.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033952729999999999 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545355+0000", 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30212512200000002 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545446+0000", 2024-09-15T21:00:45.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1941999999999995e-05 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:02.545581+0000", 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013493199999999999 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:104 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.090 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 41.670220391999997, 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30628519900000001, 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.091 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311979+0000", 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.311980+0000", 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312116+0000", 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000136129 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312137+0000", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1026000000000002e-05 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.312147+0000", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.594e-06 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316046+0000", 2024-09-15T21:00:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003899062 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316173+0000", 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012724000000000001 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.316430+0000", 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025629800000000002 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.317718+0000", 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001287949 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618117+0000", 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30039965099999999 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618203+0000", 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5415999999999997e-05 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:03.618264+0000", 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1820000000000001e-05 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.098 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.098 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:45.099 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.753930932999999, 2024-09-15T21:00:45.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:00:45.099 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:00:45.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:00:45.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:00:45.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.753760890000002, 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.106 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:00:45.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:45.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:00:45.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:00:45.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:00:45.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:240 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.884481588, 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30680979800000002, 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:45.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 240 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097718+0000", 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097721+0000", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2959999999999999e-06 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097740+0000", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8397e-05 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.097776+0000", 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6189999999999997e-05 2024-09-15T21:00:45.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.197913+0000", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10013675700000001 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198064+0000", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015111200000000001 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198219+0000", 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000154823 2024-09-15T21:00:45.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.404528+0000", 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20630922300000001 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:242 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.784180538999999, 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30841185399999999, 2024-09-15T21:00:45.117 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 242 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198019+0000", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198020+0000", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000002e-07 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198024+0000", 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4100000000000001e-06 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198035+0000", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0507e-05 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.198058+0000", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3606000000000001e-05 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.201317+0000", 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003258862 2024-09-15T21:00:45.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.201451+0000", 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013384200000000001 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.506431+0000", 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30497976700000001 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:45.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:45.122 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:140: finishing write tid 5 to smithi02732947-42 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:140: oid 42 updating version 0 to 69 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:140: oid 42 version 69 is already newer than 67 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:140: oid 42 version 69 is already newer than 68 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:140: oid 42 updating version 69 to 70 2024-09-15T21:00:45.123 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 70 (ObjNum 77 snap 9 seq_num 77) dirty exists 2024-09-15T21:00:45.124 INFO:tasks.rados.rados.0.smithi027.stdout:140: left oid 42 (ObjNum 77 snap 9 seq_num 77) 2024-09-15T21:00:45.124 INFO:tasks.rados.rados.0.smithi027.stdout:146: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T21:00:45.124 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:45.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.987+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:45.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.987+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:00:45.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.991+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4 to read 0,1,4,5 2024-09-15T21:00:45.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.991+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:00:45.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.995+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:45.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:44.995+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:00:45.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:45.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:45.095+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:00:45.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:45.095+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:00:45.127 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:45.120+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:45.127 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:45.145 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 124750 2024-09-15T21:00:45.161 INFO:tasks.rados.rados.0.smithi027.stdout:151: finishing write tid 1 to smithi02732947-4 2024-09-15T21:00:45.161 INFO:tasks.rados.rados.0.smithi027.stdout:151: finishing write tid 2 to smithi02732947-4 2024-09-15T21:00:45.161 INFO:tasks.rados.rados.0.smithi027.stdout:151: finishing write tid 3 to smithi02732947-4 2024-09-15T21:00:45.161 INFO:tasks.rados.rados.0.smithi027.stdout:151: finishing write tid 4 to smithi02732947-4 2024-09-15T21:00:45.163 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:45.160+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:45.163 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:45.160+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,4,5 2024-09-15T21:00:45.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:45.219+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPush(3.6s1 53/50 [PushOp(3:6d8f48d3:::smithi02732947-20:head, version: 34'7, data_included: [0~81920], data_size: 81920, omap_header_size: 0, omap_entries_size: 0, attrset_size: 24, recovery_info: ObjectRecoveryInfo(3:6d8f48d3:::smithi02732947-20:head@34'7, size: 327680, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:45.074895+0000", 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.182947108, 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18303423499999999, 2024-09-15T21:00:45.258 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.074895+0000", 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.074895+0000", 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.074897+0000", 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.007e-06 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.075261+0000", 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036423299999999999 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.075325+0000", 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4288000000000005e-05 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.075374+0000", 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8510999999999997e-05 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/28 ECSubWrite(tid=52, reqid=client.4641.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=53'15))", 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:45.162325+0000", 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.095516409999999996, 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095681551000000004, 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162325+0000", 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162325+0000", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162327+0000", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.165e-06 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162352+0000", 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4261e-05 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162391+0000", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9644999999999997e-05 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162408+0000", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6611000000000001e-05 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162511+0000", 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102641 2024-09-15T21:00:45.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.162525+0000", 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3891e-05 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.0s5 53/28 ECSubRead(tid=53, to_read={3:0196c8aa:::smithi02732947-4:head=0,4096,0}, subchunks={3:0196c8aa:::smithi02732947-4:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:45.163266+0000", 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.09457554, 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094827070999999999, 2024-09-15T21:00:45.265 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163266+0000", 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163266+0000", 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163267+0000", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1689999999999999e-06 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163281+0000", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3774000000000001e-05 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163320+0000", 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8651e-05 2024-09-15T21:00:45.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:45.163336+0000", 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5811000000000002e-05 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:00:45.268 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:45.269 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:45.260+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,4,5 2024-09-15T21:00:45.269 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:45.260+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,4,5 2024-09-15T21:00:45.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:45.307 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 50 v 71 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T21:00:45.358 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:45.356+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:45.364 INFO:tasks.rados.rados.0.smithi027.stdout:151: finishing write tid 5 to smithi02732947-4 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:151: oid 4 updating version 0 to 13 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:151: oid 4 version 13 is already newer than 12 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:151: oid 4 updating version 13 to 14 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:151: oid 4 updating version 14 to 15 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 15 (ObjNum 78 snap 10 seq_num 78) dirty exists 2024-09-15T21:00:45.365 INFO:tasks.rados.rados.0.smithi027.stdout:151: left oid 4 (ObjNum 78 snap 10 seq_num 78) 2024-09-15T21:00:45.426 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:45.421+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:45.437 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:45.440 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:45.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:45.527 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:45.525+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:140: done (4 left) 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:146: done (3 left) 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:147: done (2 left) 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:151: done (1 left) 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:152: done (0 left) 2024-09-15T21:00:45.542 INFO:tasks.rados.rados.0.smithi027.stdout:153: snap_create 2024-09-15T21:00:45.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.329486773999999, 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:00:45.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.328183863, 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.629 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:00:45.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:00:45.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.328103938000002, 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:00:45.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.121620089, 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.641 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:00:45.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:00:45.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.120222643999998, 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:00:45.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.120183417, 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:00:45.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:00:45.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:45.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.918332949, 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.917962133, 2024-09-15T21:00:45.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:00:45.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:00:45.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.917792470999998, 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:00:45.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:00:45.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.917384281, 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:00:45.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:00:45.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.850104436999999, 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:00:45.673 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:00:45.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:00:45.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=10, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.848949012999999, 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23004227399999999, 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772834+0000", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772835+0000", 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5230000000000001e-06 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772844+0000", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4670000000000004e-06 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.772869+0000", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4955000000000001e-05 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873000+0000", 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100131232 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.002843+0000", 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.129842652 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.002876+0000", 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3445000000000003e-05 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.818891534999999, 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:00:45.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=10, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.817768095000002, 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29916498600000002, 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804015+0000", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804017+0000", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.933e-06 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804029+0000", 2024-09-15T21:00:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2016e-05 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804059+0000", 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9890999999999999e-05 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804086+0000", 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7637999999999999e-05 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103155+0000", 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29906852699999997 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103180+0000", 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981e-05 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:45.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.817122664999999, 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.814566808999999, 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295896192, 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.692 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807216+0000", 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807218+0000", 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9360000000000002e-06 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807227+0000", 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.02e-06 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807254+0000", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7260999999999999e-05 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.807268+0000", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4005999999999999e-05 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103079+0000", 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29581095699999999 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103112+0000", 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3012000000000002e-05 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.748736935, 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23024503599999999, 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873046+0000", 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873047+0000", 2024-09-15T21:00:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0559999999999999e-06 2024-09-15T21:00:45.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873054+0000", 2024-09-15T21:00:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:00:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873072+0000", 2024-09-15T21:00:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7183000000000001e-05 2024-09-15T21:00:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873084+0000", 2024-09-15T21:00:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2645e-05 2024-09-15T21:00:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103254+0000", 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23017025499999999 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.103291+0000", 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6414000000000001e-05 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:00:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.748496037999999, 2024-09-15T21:00:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.333051763, 2024-09-15T21:00:45.705 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:45.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873287+0000", 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873288+0000", 2024-09-15T21:00:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.077e-06 2024-09-15T21:00:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873309+0000", 2024-09-15T21:00:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1588000000000001e-05 2024-09-15T21:00:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873333+0000", 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3842e-05 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.873347+0000", 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3799e-05 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206159+0000", 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.33281165400000001 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206339+0000", 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000179803 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:166 3.9s0 3:9b48fc7f:::smithi02732947-22:head [copy-from ver 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.717098184000001, 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.71698077900000001, 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.714 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:00:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:00:45.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904687+0000", 2024-09-15T21:00:45.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-15T21:00:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904696+0000", 2024-09-15T21:00:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9899999999999992e-06 2024-09-15T21:00:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904709+0000", 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2543e-05 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904717+0000", 2024-09-15T21:00:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:00:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904853+0000", 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013647299999999999 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904974+0000", 2024-09-15T21:00:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120728 2024-09-15T21:00:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115180+0000", 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210206536 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:45.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116911+0000", 2024-09-15T21:00:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730517 2024-09-15T21:00:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:45.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621588+0000", 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.504676919 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621665+0000", 2024-09-15T21:00:45.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7520000000000003e-05 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.728 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 37/28 ECSubWriteReply(tid=26, last_complete=37'13, committed=1, applied=1))", 2024-09-15T21:00:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:29.372288+0000", 2024-09-15T21:00:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.249494563999999, 2024-09-15T21:00:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.301446769, 2024-09-15T21:00:45.729 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:45.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372288+0000", 2024-09-15T21:00:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372288+0000", 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372289+0000", 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0299999999999998e-07 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372295+0000", 2024-09-15T21:00:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5899999999999998e-06 2024-09-15T21:00:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372323+0000", 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8294999999999999e-05 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:45.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.372347+0000", 2024-09-15T21:00:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3578999999999999e-05 2024-09-15T21:00:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:45.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.673698+0000", 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30135106099999998 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.673735+0000", 2024-09-15T21:00:45.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7441000000000003e-05 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:45.738 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:45.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:45.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:45.735+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:45.841 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:45.837+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:45.869 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:45.869 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:45.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:45.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:45.935+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:46.009 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:46.009 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:46.009 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:46.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:46.066 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:46.064+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:46.145 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:46.141+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 33/28 ECSubWrite(tid=17, reqid=client.4641.0:106, at_version=33'11, trim_to=0'0, roll_forward_to=32'10))", 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.280559+0000", 2024-09-15T21:00:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.86321165, 2024-09-15T21:00:46.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102499246, 2024-09-15T21:00:46.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280559+0000", 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280559+0000", 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280559+0000", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7000000000000004e-07 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280892+0000", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000332255 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280918+0000", 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6594e-05 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.280941+0000", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2425e-05 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.380865+0000", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099924383000000006 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.380889+0000", 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4331000000000001e-05 2024-09-15T21:00:46.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.382964+0000", 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002074397 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.383058+0000", 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4191000000000007e-05 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s1 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.686663+0000", 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.457107182000001, 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10276225, 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686663+0000", 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686663+0000", 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686664+0000", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686683+0000", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8955999999999999e-05 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686702+0000", 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9125000000000001e-05 2024-09-15T21:00:46.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686719+0000", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6566e-05 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686866+0000", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014699299999999999 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686890+0000", 2024-09-15T21:00:46.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4026e-05 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.789358+0000", 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102468145 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.789425+0000", 2024-09-15T21:00:46.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7508999999999993e-05 2024-09-15T21:00:46.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s1 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.686846+0000", 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.456924233999999, 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106160874, 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686846+0000", 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686846+0000", 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.686847+0000", 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9999999999999998e-07 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687166+0000", 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031953299999999999 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687183+0000", 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6444999999999999e-05 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687194+0000", 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.787623+0000", 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100429185 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.787636+0000", 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3003000000000001e-05 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792929+0000", 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052934790000000002 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.793007+0000", 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7692000000000001e-05 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.2s1 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.687217+0000", 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.456553538000001, 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10798811, 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687217+0000", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687217+0000", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687218+0000", 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687236+0000", 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8179e-05 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687247+0000", 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1501999999999999e-05 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687258+0000", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.025e-05 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.788884+0000", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101626704 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.788912+0000", 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7328e-05 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.795128+0000", 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0062166390000000004 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.795205+0000", 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6817999999999995e-05 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.es5 33/28 ECSubWrite(tid=18, reqid=client.4641.0:143, at_version=33'4, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.687380+0000", 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.456390535000001, 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10500213899999999, 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687380+0000", 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687380+0000", 2024-09-15T21:00:46.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687381+0000", 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3260000000000001e-06 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687455+0000", 2024-09-15T21:00:46.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3836000000000003e-05 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687477+0000", 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2018000000000002e-05 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.687489+0000", 2024-09-15T21:00:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1489e-05 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.787680+0000", 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100191426 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.787694+0000", 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4511999999999999e-05 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792297+0000", 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046027940000000003 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792382+0000", 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4738000000000001e-05 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 33/28 ECSubRead(tid=24, to_read={3:29dd4e88:::smithi02732947-38:head=0,4096,0}, subchunks={3:29dd4e88:::smithi02732947-38:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.651103353, 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20085125200000001, 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492668+0000", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492688+0000", 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9755999999999999e-05 2024-09-15T21:00:46.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492721+0000", 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2561000000000002e-05 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.592869+0000", 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100148689 2024-09-15T21:00:46.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693164+0000", 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10029479600000001 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693518+0000", 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035428 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.028225222, 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20249605200000001, 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.182 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115546+0000", 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5900000000000005e-07 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115662+0000", 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011552699999999999 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115676+0000", 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4547e-05 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215777+0000", 2024-09-15T21:00:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100100516 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215874+0000", 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7094000000000002e-05 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215886+0000", 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759999999999999e-05 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.317999+0000", 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102113494 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318041+0000", 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2154999999999999e-05 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s1 34/28 ECSubWrite(tid=25, reqid=client.4641.0:179, at_version=34'6, trim_to=0'0, roll_forward_to=31'4))", 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.319644+0000", 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.824126742000001, 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10597272100000001, 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319644+0000", 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319644+0000", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319644+0000", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9299999999999998e-07 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319770+0000", 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000125742 2024-09-15T21:00:46.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319790+0000", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9639999999999998e-05 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.419887+0000", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100097192 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.420056+0000", 2024-09-15T21:00:46.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169063 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.420072+0000", 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5909e-05 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.425568+0000", 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054961150000000002 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.425616+0000", 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8467000000000002e-05 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 38/28 ECSubWrite(tid=70, reqid=osd.5.0:70, at_version=38'12, trim_to=0'0, roll_forward_to=37'10))", 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:30.079672+0000", 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.064098861000002, 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10406887099999999, 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079672+0000", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079672+0000", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079672+0000", 2024-09-15T21:00:46.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079812+0000", 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013927199999999999 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.079839+0000", 2024-09-15T21:00:46.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7671999999999999e-05 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.179949+0000", 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100110185 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180132+0000", 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018238600000000001 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180164+0000", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1692000000000001e-05 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.183686+0000", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035226939999999998 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.183740+0000", 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4193000000000001e-05 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.3s6 38/28 ECSubWrite(tid=15, reqid=osd.4.0:15, at_version=38'12, trim_to=0'0, roll_forward_to=36'10))", 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:30.180124+0000", 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.963646408000001, 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104168936, 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180124+0000", 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180124+0000", 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180125+0000", 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7000000000000005e-07 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180179+0000", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4824000000000002e-05 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.180209+0000", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9544e-05 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280316+0000", 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100106692 2024-09-15T21:00:46.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280485+0000", 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000169235 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280510+0000", 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5273999999999999e-05 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.284180+0000", 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0036699430000000002 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.284293+0000", 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000112854 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 38/28 ECSubWrite(tid=29, reqid=osd.8.0:29, at_version=38'15, trim_to=0'0, roll_forward_to=37'13))", 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:30.280424+0000", 2024-09-15T21:00:46.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.863346537, 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10391668, 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280424+0000", 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280424+0000", 2024-09-15T21:00:46.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280425+0000", 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6000000000000003e-07 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280479+0000", 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4784000000000003e-05 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.280508+0000", 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8711e-05 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.380644+0000", 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100136191 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.380779+0000", 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000134373 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.380793+0000", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4866e-05 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.384250+0000", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00345642 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.384341+0000", 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0674999999999995e-05 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 38/28 ECSubWrite(tid=30, reqid=osd.8.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=38'15))", 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:30.385347+0000", 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.758423430000001, 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10268434899999999, 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385347+0000", 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385347+0000", 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385349+0000", 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385374+0000", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5369000000000001e-05 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385409+0000", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5348000000000001e-05 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385434+0000", 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4379000000000002e-05 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385627+0000", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000192859 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.385658+0000", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0961999999999999e-05 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.487942+0000", 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102284307 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:30.488031+0000", 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9435e-05 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 39/28 ECSubWrite(tid=32, reqid=osd.8.0:31, at_version=0'0, trim_to=0'0, roll_forward_to=39'17))", 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.309533+0000", 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.834237561, 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102526986, 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.309533+0000", 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.309533+0000", 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.309535+0000", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.309565+0000", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0080999999999999e-05 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.309597+0000", 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1854999999999999e-05 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.409692+0000", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10009476 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.409829+0000", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137486 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.409844+0000", 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4932e-05 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.411979+0000", 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021352200000000002 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.412060+0000", 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0531999999999994e-05 2024-09-15T21:00:46.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.770617172, 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:00:46.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:00:46.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.670137088000001, 2024-09-15T21:00:46.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20147183599999999, 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473634+0000", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473665+0000", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473707+0000", 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1860999999999998e-05 2024-09-15T21:00:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473724+0000", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7206000000000001e-05 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673974+0000", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20025005600000001 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673991+0000", 2024-09-15T21:00:46.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6368999999999999e-05 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675017+0000", 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001026267 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675105+0000", 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8195000000000005e-05 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=42, reqid=client.4641.0:265, at_version=39'30, trim_to=0'0, roll_forward_to=39'28))", 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.934901+0000", 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.208869221, 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10236883500000001, 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934901+0000", 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934901+0000", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934903+0000", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5629999999999999e-06 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934943+0000", 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0351e-05 2024-09-15T21:00:46.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.934974+0000", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0584999999999999e-05 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.035137+0000", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100162958 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.035332+0000", 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019525599999999999 2024-09-15T21:00:46.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.035354+0000", 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2385999999999999e-05 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037194+0000", 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018393800000000001 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037270+0000", 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6355999999999999e-05 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 48/28 ECSubWrite(tid=44, reqid=osd.8.0:43, at_version=0'0, trim_to=0'0, roll_forward_to=48'27))", 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.261607+0000", 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.882163126, 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103300145, 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261607+0000", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261607+0000", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261609+0000", 2024-09-15T21:00:46.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2640000000000001e-06 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261625+0000", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6739999999999999e-05 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261646+0000", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0355000000000001e-05 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.361725+0000", 2024-09-15T21:00:46.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100079553 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.361790+0000", 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5203999999999994e-05 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.361800+0000", 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8989999999999996e-06 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.364883+0000", 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030823589999999998 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.364907+0000", 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4771000000000001e-05 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=41, reqid=client.4641.0:360, at_version=48'16, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:00:46.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5908693339999997, 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20315024000000001, 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552903+0000", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.561e-06 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553100+0000", 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019769300000000001 2024-09-15T21:00:46.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553123+0000", 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3008999999999999e-05 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653267+0000", 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014326899999999 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753534+0000", 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100267523 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753550+0000", 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5824999999999998e-05 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755996+0000", 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024464930000000001 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756051+0000", 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4867000000000002e-05 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=42, reqid=client.4641.0:361, at_version=48'17, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:00:46.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.653327+0000", 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4904429969999997, 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102924405, 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653327+0000", 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653327+0000", 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653329+0000", 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6929999999999999e-06 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653443+0000", 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113652 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653468+0000", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4831999999999999e-05 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653481+0000", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3129000000000001e-05 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.754463+0000", 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100982765 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.754489+0000", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5259e-05 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756215+0000", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00172668 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756252+0000", 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6395e-05 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=43, reqid=client.4641.0:362, at_version=48'18, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.653505+0000", 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4902652520000004, 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103435991, 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:46.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653505+0000", 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653505+0000", 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653506+0000", 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653530+0000", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3966000000000001e-05 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653541+0000", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1736000000000001e-05 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653550+0000", 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8189999999999998e-06 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755701+0000", 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10215091599999999 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:46.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755717+0000", 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5369000000000001e-05 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756868+0000", 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011511749999999999 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:46.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756941+0000", 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3449999999999996e-05 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:46.249 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:46.249 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:46.249 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:46.250 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:00:46.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:00:46.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:46.255 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:46.247+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:46.354 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:46.351+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:46.632 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:00:46.632 INFO:tasks.rados.rados.0.smithi027.stdout:153: done (0 left) 2024-09-15T21:00:46.632 INFO:tasks.rados.rados.0.smithi027.stdout:154: snap_remove snap 10 2024-09-15T21:00:46.634 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:00:46.634 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:46.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:46.636 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 55, 2024-09-15T21:00:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "07b66685-24ed-4325-b3d4-b0cceb7d7e6d", 2024-09-15T21:00:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T20:59:43.863405+0000", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T21:00:46.535557+0000", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T20:59:57.421497+0000", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T20:59:47.394909+0000", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5804032, 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "nodeep-scrub", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 6, 2024-09-15T21:00:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 12, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T21:00:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T20:59:55.427902+0000", 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:00:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:00:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "20", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:00:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:00:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T21:00:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 11.760000228881836, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 11.760000228881836, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.17000000178813934, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T21:00:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T21:00:04.483565+0000", 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T21:00:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:00:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "24", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T21:00:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 24, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:00:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:00:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 3, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 3, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.75, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2024-09-15T21:00:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "unique_pool_0", 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T21:00:17.994699+0000", 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 3, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 8, 2024-09-15T21:00:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 5, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 1, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 16, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 16, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 16, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 16, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 16, 2024-09-15T21:00:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "55", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:00:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 13, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 55, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:00:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "lrcprofile", 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:00:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 16384, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": true, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "rados": {} 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd", 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 50, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.661 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/630056637", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/630056637", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/630056637", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/630056637", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "43960b2a-b250-45e3-b725-484c0751590d", 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.662 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 0.3756866455078125, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.667 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/2797309965", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/2797309965", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/2797309965", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/2797309965", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "9653f0df-bf71-4a94-b79a-498f067c8de8", 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.668 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 19, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 21, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6825/1130579408", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6827/1130579408", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/1130579408", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/1130579408", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.673 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "9ac2adf5-f248-4838-9bd5-57d13dd53309", 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 50, 2024-09-15T21:00:46.674 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.678 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/2217694359", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/2217694359", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/2217694359", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/2217694359", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "72c3acd6-6e64-4853-8823-947f944b96b5", 2024-09-15T21:00:46.679 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6800", 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6801", 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6802", 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6803", 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6806", 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6807", 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6804", 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6805", 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6801/2520277109", 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6803/2520277109", 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6807/2520277109", 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6805/2520277109", 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ad64f860-44e7-45a6-bc0d-e85458d80f39", 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 50, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6816", 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6817", 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6818", 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6819", 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6822", 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6823", 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6820", 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6821", 2024-09-15T21:00:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6817/1665320626", 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6819/1665320626", 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6823/1665320626", 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6821/1665320626", 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "4701a955-3643-4210-800d-7a86ba5155bc", 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 50, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6808", 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6809", 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6810", 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6811", 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6814", 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6815", 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6812", 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6813", 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6809/1563776909", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6811/1563776909", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6815/1563776909", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6813/1563776909", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "abc724fc-25c2-4b2e-b4ba-16b61dbaf89d", 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6824", 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6825", 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6826", 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6827", 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6830", 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6831", 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6828", 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6829", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6825/3067780245", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6827/3067780245", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6831/3067780245", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6829/3067780245", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 8, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "b114589d-5020-4c1c-ad76-f07231e58d47", 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 49, 2024-09-15T21:00:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6824", 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6825", 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6826", 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6827", 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6830", 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6831", 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6828", 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6829", 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6825/437424413", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6827/437424413", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6831/437424413", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6829/437424413", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 9, 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "6e4c938a-4505-41f5-90dc-416c6ceb868a", 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 0, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6800", 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6801", 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6802", 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6803", 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6806", 2024-09-15T21:00:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6807", 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6804", 2024-09-15T21:00:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6805", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6801/591511703", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6803/591511703", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6807/591511703", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6805/591511703", 2024-09-15T21:00:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 10, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "fa725c67-48fe-45ac-bcfb-6cf3fddea7fe", 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 49, 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6808", 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6809", 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6810", 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6811", 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6814", 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6815", 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6812", 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6813", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6809/2398207120", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6811/2398207120", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6815/2398207120", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6813/2398207120", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 11, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "c6fcefe9-5105-45bf-ac2d-ca8eb186316c", 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 53, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:00:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6816", 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6817", 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6818", 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6819", 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6822", 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6823", 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6820", 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6821", 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6817/2517738961", 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6819/2517738961", 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6823/2517738961", 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6821/2517738961", 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.788074+0000", 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.674356+0000", 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.389412+0000", 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.966270+0000", 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.604135+0000", 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.110212+0000", 2024-09-15T21:00:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.787136+0000", 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2024-09-15T21:00:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.668403+0000", 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 8, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.568433+0000", 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 9, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.645022+0000", 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 10, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.965253+0000", 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 11, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:00:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.847328+0000", 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [ 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.4", 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 11, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 9, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 5, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 2, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 6, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 0 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.5", 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 5, 2024-09-15T21:00:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: 3, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 6, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 7, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 11, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 10 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.6", 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 0, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: 4, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 7, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 10, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 9, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 5 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.a", 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 6, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 4, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 10, 2024-09-15T21:00:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 8, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 9, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 3, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 7 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "3.b", 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 3, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 0, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 4, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 8, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 11, 2024-09-15T21:00:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 7, 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: 6, 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: 2147483647 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T21:00:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "lrcprofile": { 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-device-class": "", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-num-failure-domains": "0", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-osds-per-failure-domain": "0", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-root": "default", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "4", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "l": "3", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "2", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "lrcprofile", 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "lrc" 2024-09-15T21:00:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [ 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 3, 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 6, 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 5 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:46.742 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:46.743 INFO:tasks.thrashosds.thrasher:j is {'epoch': 55, 'fsid': '07b66685-24ed-4325-b3d4-b0cceb7d7e6d', 'created': '2024-09-15T20:59:43.863405+0000', 'modified': '2024-09-15T21:00:46.535557+0000', 'last_up_change': '2024-09-15T20:59:57.421497+0000', 'last_in_change': '2024-09-15T20:59:47.394909+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T20:59:55.427902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 11.760000228881836, 'score_stable': 11.760000228881836, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:00:04.483565+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T21:00:17.994699+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 8, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 55, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'lrcprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': True, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'd1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 630056637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 630056637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 630056637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 630056637}]}, 'public_addr': '172.21.15.27:6809/630056637', 'cluster_addr': '172.21.15.27:6811/630056637', 'heartbeat_back_addr': '172.21.15.27:6815/630056637', 'heartbeat_front_addr': '172.21.15.27:6813/630056637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '43960b2a-b250-45e3-b725-484c0751590d', 'up': 1, 'in': 1, 'weight': 0.3756866455078125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 2797309965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 2797309965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 2797309965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 2797309965}]}, 'public_addr': '172.21.15.27:6801/2797309965', 'cluster_addr': '172.21.15.27:6803/2797309965', 'heartbeat_back_addr': '172.21.15.27:6807/2797309965', 'heartbeat_front_addr': '172.21.15.27:6805/2797309965', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9653f0df-bf71-4a94-b79a-498f067c8de8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 19, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6824', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6825', 'nonce': 1130579408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6826', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6827', 'nonce': 1130579408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6830', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6831', 'nonce': 1130579408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6828', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6829', 'nonce': 1130579408}]}, 'public_addr': '172.21.15.27:6825/1130579408', 'cluster_addr': '172.21.15.27:6827/1130579408', 'heartbeat_back_addr': '172.21.15.27:6831/1130579408', 'heartbeat_front_addr': '172.21.15.27:6829/1130579408', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ac2adf5-f248-4838-9bd5-57d13dd53309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 2217694359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 2217694359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 2217694359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 2217694359}]}, 'public_addr': '172.21.15.27:6817/2217694359', 'cluster_addr': '172.21.15.27:6819/2217694359', 'heartbeat_back_addr': '172.21.15.27:6823/2217694359', 'heartbeat_front_addr': '172.21.15.27:6821/2217694359', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '72c3acd6-6e64-4853-8823-947f944b96b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6800', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6801', 'nonce': 2520277109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6802', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6803', 'nonce': 2520277109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6806', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6807', 'nonce': 2520277109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6804', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6805', 'nonce': 2520277109}]}, 'public_addr': '172.21.15.66:6801/2520277109', 'cluster_addr': '172.21.15.66:6803/2520277109', 'heartbeat_back_addr': '172.21.15.66:6807/2520277109', 'heartbeat_front_addr': '172.21.15.66:6805/2520277109', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ad64f860-44e7-45a6-bc0d-e85458d80f39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6816', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6817', 'nonce': 1665320626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6818', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6819', 'nonce': 1665320626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6822', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6823', 'nonce': 1665320626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6820', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6821', 'nonce': 1665320626}]}, 'public_addr': '172.21.15.66:6817/1665320626', 'cluster_addr': '172.21.15.66:6819/1665320626', 'heartbeat_back_addr': '172.21.15.66:6823/1665320626', 'heartbeat_front_addr': '172.21.15.66:6821/1665320626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4701a955-3643-4210-800d-7a86ba5155bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6808', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6809', 'nonce': 1563776909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6810', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6811', 'nonce': 1563776909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6814', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6815', 'nonce': 1563776909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6812', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6813', 'nonce': 1563776909}]}, 'public_addr': '172.21.15.66:6809/1563776909', 'cluster_addr': '172.21.15.66:6811/1563776909', 'heartbeat_back_addr': '172.21.15.66:6815/1563776909', 'heartbeat_front_addr': '172.21.15.66:6813/1563776909', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'abc724fc-25c2-4b2e-b4ba-16b61dbaf89d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6824', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6825', 'nonce': 3067780245}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6826', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6827', 'nonce': 3067780245}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6830', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6831', 'nonce': 3067780245}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6828', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6829', 'nonce': 3067780245}]}, 'public_addr': '172.21.15.66:6825/3067780245', 'cluster_addr': '172.21.15.66:6827/3067780245', 'heartbeat_back_addr': '172.21.15.66:6831/3067780245', 'heartbeat_front_addr': '172.21.15.66:6829/3067780245', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'b114589d-5020-4c1c-ad76-f07231e58d47', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 437424413}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 437424413}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 437424413}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 437424413}]}, 'public_addr': '172.21.15.92:6825/437424413', 'cluster_addr': '172.21.15.92:6827/437424413', 'heartbeat_back_addr': '172.21.15.92:6831/437424413', 'heartbeat_front_addr': '172.21.15.92:6829/437424413', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '6e4c938a-4505-41f5-90dc-416c6ceb868a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 591511703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 591511703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 591511703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 591511703}]}, 'public_addr': '172.21.15.92:6801/591511703', 'cluster_addr': '172.21.15.92:6803/591511703', 'heartbeat_back_addr': '172.21.15.92:6807/591511703', 'heartbeat_front_addr': '172.21.15.92:6805/591511703', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'fa725c67-48fe-45ac-bcfb-6cf3fddea7fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2398207120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2398207120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2398207120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2398207120}]}, 'public_addr': '172.21.15.92:6809/2398207120', 'cluster_addr': '172.21.15.92:6811/2398207120', 'heartbeat_back_addr': '172.21.15.92:6815/2398207120', 'heartbeat_front_addr': '172.21.15.92:6813/2398207120', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'c6fcefe9-5105-45bf-ac2d-ca8eb186316c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2517738961}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 2517738961}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 2517738961}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 2517738961}]}, 'public_addr': '172.21.15.92:6817/2517738961', 'cluster_addr': '172.21.15.92:6819/2517738961', 'heartbeat_back_addr': '172.21.15.92:6823/2517738961', 'heartbeat_front_addr': '172.21.15.92:6821/2517738961', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.788074+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.674356+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.389412+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.966270+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.604135+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.110212+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.787136+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.668403+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.568433+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.645022+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.965253+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.847328+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.4', 'osds': [2147483647, 11, 9, 5, 2147483647, 2, 6, 0]}, {'pgid': '3.5', 'osds': [5, 3, 2, 6, 7, 2147483647, 11, 10]}, {'pgid': '3.6', 'osds': [0, 2147483647, 4, 7, 2147483647, 10, 9, 5]}, {'pgid': '3.a', 'osds': [6, 4, 2147483647, 10, 8, 9, 3, 7]}, {'pgid': '3.b', 'osds': [3, 0, 4, 8, 11, 7, 6, 2147483647]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'lrcprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'l': '3', 'm': '2', 'name': 'lrcprofile', 'plugin': 'lrc'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 3, 'length': 1}, {'begin': 6, 'length': 5}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:00:46.744 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T21:00:46.744 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:46.629+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:46.745 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:46.744+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:46.846 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:46.844+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:46.878 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:46.878 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:46.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:46.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:46.974 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:46.973+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:47.048 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:47.043+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:47.064 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:47.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.8s6 33/28 ECSubWrite(tid=22, reqid=client.4641.0:154, at_version=33'3, trim_to=0'0, roll_forward_to=30'2))", 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.748880240999998, 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20195380099999999, 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312925+0000", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312927+0000", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3e-06 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313031+0000", 2024-09-15T21:00:47.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000104053 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313061+0000", 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0015e-05 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313085+0000", 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3127999999999999e-05 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.513445+0000", 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20036016600000001 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.513463+0000", 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8499e-05 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.514821+0000", 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013581699999999999 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.514879+0000", 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7469999999999997e-05 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/28 ECSubWrite(tid=39, reqid=client.4641.0:152, at_version=33'17, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.048533083999999, 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20457245199999999, 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:47.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013272+0000", 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013273+0000", 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4799999999999997e-07 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013291+0000", 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7527e-05 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.013303+0000", 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2486e-05 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116075+0000", 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102772293 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216396+0000", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100321059 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216405+0000", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8359999999999998e-06 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217565+0000", 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001159481 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217845+0000", 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027992199999999999 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 33/28 ECSubWrite(tid=40, reqid=client.4641.0:156, at_version=33'18, trim_to=0'0, roll_forward_to=33'14))", 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.945685493999999, 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20296982799999999, 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.074 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116120+0000", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116121+0000", 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116221+0000", 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010067 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116235+0000", 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4117000000000001e-05 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.216815+0000", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100580146 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316782+0000", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099966606 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316815+0000", 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3225000000000001e-05 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319037+0000", 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022215490000000002 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.319090+0000", 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s7 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.94549365, 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20252270999999999, 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116312+0000", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116482+0000", 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017011500000000001 2024-09-15T21:00:47.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116500+0000", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7708999999999999e-05 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.217229+0000", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100729376 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316421+0000", 2024-09-15T21:00:47.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099191814000000003 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.316431+0000", 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7319999999999993e-06 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318772+0000", 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023414769999999998 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318834+0000", 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2162999999999994e-05 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s7 37/28 ECSubRead(tid=28, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.385620705000001, 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20065597199999999, 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676185+0000", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676186+0000", 2024-09-15T21:00:47.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7430000000000001e-06 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676208+0000", 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1985e-05 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.676247+0000", 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8852e-05 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.776372+0000", 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10012509 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.876689+0000", 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100317034 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:29.876841+0000", 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151268 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=76, reqid=client.4641.0:244, at_version=39'36, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.958375496, 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20322754300000001, 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:47.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103430+0000", 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103431+0000", 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4700000000000001e-07 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103932+0000", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000501845 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103966+0000", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3621999999999998e-05 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.103990+0000", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3739000000000001e-05 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.204334+0000", 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100344055 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.204352+0000", 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8236e-05 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.306570+0000", 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10221767 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.306657+0000", 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7628999999999997e-05 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=77, reqid=osd.5.0:77, at_version=39'38, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:47.091 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.954348166000001, 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20007127199999999, 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107457+0000", 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107459+0000", 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2790000000000001e-06 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107499+0000", 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9437999999999997e-05 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107531+0000", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1862000000000001e-05 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.107554+0000", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3569000000000001e-05 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.305963+0000", 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19840924300000001 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.305979+0000", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5248999999999999e-05 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.307408+0000", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001428868 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.307528+0000", 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120764 2024-09-15T21:00:47.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 39/28 ECSubWrite(tid=33, reqid=client.4641.0:256, at_version=39'11, trim_to=0'0, roll_forward_to=34'7))", 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.833685085000001, 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104120988, 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228120+0000", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228122+0000", 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0590000000000001e-06 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228164+0000", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2011000000000001e-05 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228204+0000", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9780999999999997e-05 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.228227+0000", 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3405999999999999e-05 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.328399+0000", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100171435 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.328423+0000", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4485e-05 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.332178+0000", 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0037545349999999998 2024-09-15T21:00:47.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.332241+0000", 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3275999999999995e-05 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 39/28 ECSubWrite(tid=78, reqid=client.4641.0:257, at_version=39'40, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.748936103, 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20214031900000001, 2024-09-15T21:00:47.100 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312869+0000", 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312871+0000", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.79e-06 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312934+0000", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2856999999999998e-05 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.312974+0000", 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9790999999999999e-05 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413116+0000", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014285000000001 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513330+0000", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100213416 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513346+0000", 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5852e-05 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514911+0000", 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001565108 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.515009+0000", 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8654999999999996e-05 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s4 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.688542204000001, 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.201922037, 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373263+0000", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373265+0000", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.199e-06 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373383+0000", 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000117367 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373415+0000", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1931999999999999e-05 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373430+0000", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5894999999999999e-05 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473678+0000", 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100247579 2024-09-15T21:00:47.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473714+0000", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5858000000000001e-05 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575119+0000", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101405157 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575185+0000", 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6050000000000006e-05 2024-09-15T21:00:47.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s4 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.684657787000001, 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19810077700000001, 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377147+0000", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377149+0000", 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.787e-06 2024-09-15T21:00:47.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377181+0000", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1494e-05 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377211+0000", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0015e-05 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.377226+0000", 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5282999999999999e-05 2024-09-15T21:00:47.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.474792+0000", 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.097565747999999994 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.474821+0000", 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9697999999999999e-05 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575194+0000", 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100372138 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575248+0000", 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4614000000000003e-05 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 39/28 ECSubWrite(tid=79, reqid=osd.5.0:79, at_version=39'14, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:47.113 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.648635268, 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16219983700000001, 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413170+0000", 2024-09-15T21:00:47.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413171+0000", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3170000000000001e-06 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413238+0000", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6926000000000003e-05 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413269+0000", 2024-09-15T21:00:47.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0491e-05 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413292+0000", 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2812000000000001e-05 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513803+0000", 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10051183 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513828+0000", 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5116000000000001e-05 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575263+0000", 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061434158000000003 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575370+0000", 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107187 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 39/28 ECSubWrite(tid=80, reqid=client.4641.0:250, at_version=39'15, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.648481540000001, 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16208093700000001, 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:47.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413324+0000", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413325+0000", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413482+0000", 2024-09-15T21:00:47.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000157262 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.413501+0000", 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9398999999999998e-05 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.513640+0000", 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100138827 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514646+0000", 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010056279999999999 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.514676+0000", 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9808e-05 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575374+0000", 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060698479 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.575405+0000", 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0550999999999999e-05 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 39/28 ECSubWrite(tid=34, reqid=client.4641.0:256, at_version=0'0, trim_to=0'0, roll_forward_to=39'11))", 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.431067584000001, 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12595379500000001, 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630738+0000", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630740+0000", 2024-09-15T21:00:47.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8369999999999999e-06 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630759+0000", 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9888999999999999e-05 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630790+0000", 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.630816+0000", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5664999999999999e-05 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.731158+0000", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100342483 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.731174+0000", 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5463000000000001e-05 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.756611+0000", 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025437517999999999 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.756691+0000", 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0072000000000002e-05 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.cs3 39/28 ECSubRead(tid=87, to_read={3:3f0b4dcc:::smithi02732947-9:head=0,4096,0}, subchunks={3:3f0b4dcc:::smithi02732947-9:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:47.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.183492511000001, 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.158771838, 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878313+0000", 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878315+0000", 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7829999999999999e-06 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878334+0000", 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9043e-05 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.878384+0000", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0587000000000002e-05 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.978506+0000", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100122275 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.036912+0000", 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058405743000000003 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:32.037085+0000", 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017240699999999999 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s7 48/28 ECSubWrite(tid=43, reqid=osd.8.0:43, at_version=48'27, trim_to=0'0, roll_forward_to=46'25))", 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.144963+0000", 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9168418430000003, 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11613765600000001, 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.144963+0000", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.144963+0000", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.144964+0000", 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.145019+0000", 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.469e-05 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.145055+0000", 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6596e-05 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.245172+0000", 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10011703199999999 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.255740+0000", 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010567607999999999 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.255751+0000", 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1105e-05 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261060+0000", 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053093710000000002 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.261101+0000", 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0664000000000002e-05 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.1s2 48/28 ECSubWrite(tid=38, reqid=client.4641.0:354, at_version=48'7, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:47.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.154771+0000", 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9070344690000001, 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10574312700000001, 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154771+0000", 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154771+0000", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154772+0000", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.173e-06 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154812+0000", 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9854999999999997e-05 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154844+0000", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1742000000000002e-05 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.154864+0000", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0193000000000001e-05 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.255241+0000", 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10037771500000001 2024-09-15T21:00:47.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.255253+0000", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1467999999999999e-05 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.260485+0000", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052317780000000003 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.260514+0000", 2024-09-15T21:00:47.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9203000000000001e-05 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s5 48/28 ECSubWrite(tid=42, reqid=client.4641.0:361, at_version=48'17, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.558436+0000", 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.5033689609999996, 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20081449500000001, 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.558436+0000", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.558436+0000", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.558437+0000", 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.37e-07 2024-09-15T21:00:47.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.558519+0000", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2032000000000003e-05 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.558544+0000", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5077999999999999e-05 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.658644+0000", 2024-09-15T21:00:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100099936 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.658914+0000", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026991899999999997 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.658943+0000", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9272e-05 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.759155+0000", 2024-09-15T21:00:47.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100211518 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.759251+0000", 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6102999999999998e-05 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 48/28 ECSubWrite(tid=136, reqid=osd.5.0:129, at_version=0'0, trim_to=0'0, roll_forward_to=48'31))", 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.662216+0000", 2024-09-15T21:00:47.144 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.3995889999999997, 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20045955500000001, 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662216+0000", 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662216+0000", 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662218+0000", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9130000000000001e-06 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662245+0000", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7115999999999999e-05 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662274+0000", 2024-09-15T21:00:47.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8849000000000001e-05 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.662293+0000", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.663305+0000", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101169 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.663321+0000", 2024-09-15T21:00:47.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.562e-05 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.862618+0000", 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19929702099999999 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.862676+0000", 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8010000000000002e-05 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs3 49/28 ECSubWrite(tid=137, reqid=osd.5.0:137, at_version=49'33, trim_to=0'0, roll_forward_to=48'31))", 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:41.170781+0000", 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8910244379999996, 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.201722809, 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170781+0000", 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170781+0000", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170782+0000", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170819+0000", 2024-09-15T21:00:47.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7628e-05 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170838+0000", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9126e-05 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.170854+0000", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.525e-05 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.271433+0000", 2024-09-15T21:00:47.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100579011 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.271447+0000", 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3828999999999999e-05 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.372447+0000", 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101000141 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:47.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:41.372504+0000", 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6901000000000003e-05 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.153 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:47.154 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:47.155 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:47.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:47.156 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:47.157 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:47.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:47.151+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:47.162 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:47.185 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:47.207 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:47.207 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:47.229 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:47.229 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:47.251 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:47.255 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.251+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:47.274 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:47.275 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:47.286 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.4s1 55/50 [PushReplyOp(3:2b54c2df:::smithi02732947-23:8)])", 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:47.261762+0000", 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.025009070000000001, 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.025044795000000002, 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261762+0000", 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261762+0000", 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261764+0000", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.81e-06 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261774+0000", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0108999999999999e-05 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261798+0000", 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4202999999999998e-05 2024-09-15T21:00:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:47.261813+0000", 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5333e-05 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-15T21:00:47.290 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:47.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:47.299 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:47.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:47.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:47.323 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:47.356 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:47.352+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:47.424 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:47.424 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:47.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:47.457 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:47.453+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:47.549 INFO:tasks.rados.rados.0.smithi027.stdout:154: done (0 left) 2024-09-15T21:00:47.550 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 22 current snap is 12 2024-09-15T21:00:47.550 INFO:tasks.rados.rados.0.smithi027.stdout:155: seq_num 79 ranges {786432=573440,1359872=425984,1785856=344064} 2024-09-15T21:00:47.561 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.555+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:47.573 INFO:tasks.rados.rados.0.smithi027.stdout:155: writing smithi02732947-22 from 786432 to 1359872 tid 1 2024-09-15T21:00:47.581 INFO:tasks.rados.rados.0.smithi027.stdout:155: writing smithi02732947-22 from 1359872 to 1785856 tid 2 2024-09-15T21:00:47.587 INFO:tasks.rados.rados.0.smithi027.stdout:155: writing smithi02732947-22 from 1785856 to 2129920 tid 3 2024-09-15T21:00:47.587 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 10 current snap is 12 2024-09-15T21:00:47.587 INFO:tasks.rados.rados.0.smithi027.stdout:156: seq_num 80 ranges {638976=589824,1228800=573440,1802240=196608} 2024-09-15T21:00:47.606 INFO:tasks.rados.rados.0.smithi027.stdout:156: writing smithi02732947-10 from 638976 to 1228800 tid 1 2024-09-15T21:00:47.613 INFO:tasks.rados.rados.0.smithi027.stdout:156: writing smithi02732947-10 from 1228800 to 1802240 tid 2 2024-09-15T21:00:47.614 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:47.614 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:47.616 INFO:tasks.rados.rados.0.smithi027.stdout:156: writing smithi02732947-10 from 1802240 to 1998848 tid 3 2024-09-15T21:00:47.616 INFO:tasks.rados.rados.0.smithi027.stdout:157: setattr oid 6 current snap is 12 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:158: read oid 32 snap 9 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:158: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:159: read oid 20 snap -1 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:159: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:160: rollback oid 31 current snap is 12 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 31 to 11 2024-09-15T21:00:47.618 INFO:tasks.rados.rados.0.smithi027.stdout:161: snap_create 2024-09-15T21:00:47.619 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.619 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.620 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.620 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.621 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.621 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.617+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:00:47.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:47.659 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.655+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:47.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.719+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:47.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.719+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:00:47.760 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:47.757+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:47.769 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:47.769 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:47.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:47.825 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.819+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:47.874 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:47.873+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:47.884 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 17 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T21:00:47.926 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.919+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6 to read 0,1,4,5 2024-09-15T21:00:47.926 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.919+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:00:47.926 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.919+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:00:47.930 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.923+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:47.930 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.923+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6 to read 0,1,4,5 2024-09-15T21:00:47.930 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.923+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:00:47.930 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:47.923+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:00:47.959 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:47.959 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:47.961 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:47.957+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:47.974 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:48.003 INFO:tasks.rados.rados.0.smithi027.stdout:155: finishing write tid 1 to smithi02732947-22 2024-09-15T21:00:48.004 INFO:tasks.rados.rados.0.smithi027.stdout:155: finishing write tid 2 to smithi02732947-22 2024-09-15T21:00:48.033 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:48.027+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6 to read 0,1,4,5 2024-09-15T21:00:48.034 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:48.027+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:00:48.038 INFO:tasks.rados.rados.0.smithi027.stdout:158: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-15T21:00:48.038 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 40922 2024-09-15T21:00:48.045 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:48.052 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:48.062 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:48.060+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:48.106 INFO:tasks.rados.rados.0.smithi027.stdout:155: finishing write tid 3 to smithi02732947-22 2024-09-15T21:00:48.107 INFO:tasks.rados.rados.0.smithi027.stdout:155: finishing write tid 4 to smithi02732947-22 2024-09-15T21:00:48.133 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.133 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.148 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:48.163 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.160+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:48.207 INFO:tasks.rados.rados.0.smithi027.stdout:156: finishing write tid 1 to smithi02732947-10 2024-09-15T21:00:48.208 INFO:tasks.rados.rados.0.smithi027.stdout:156: finishing write tid 2 to smithi02732947-10 2024-09-15T21:00:48.263 INFO:tasks.rados.rados.0.smithi027.stdout:156: finishing write tid 3 to smithi02732947-10 2024-09-15T21:00:48.264 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:48.259+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:48.312 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.308+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:48.312 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.308+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:48.312 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.308+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:48.312 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.308+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: finishing write tid 5 to smithi02732947-22 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: oid 22 updating version 0 to 35 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: oid 22 version 35 is already newer than 32 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: oid 22 version 35 is already newer than 33 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: oid 22 version 35 is already newer than 34 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 35 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T21:00:48.313 INFO:tasks.rados.rados.0.smithi027.stdout:155: left oid 22 (ObjNum 79 snap 12 seq_num 79) 2024-09-15T21:00:48.321 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.322 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:48.362 INFO:tasks.rados.rados.0.smithi027.stdout:160: finishing rollback tid 0 to smithi02732947-31 2024-09-15T21:00:48.363 INFO:tasks.rados.rados.0.smithi027.stdout:160: finishing rollback tid 1 to smithi02732947-31 2024-09-15T21:00:48.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:48.359+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:48.463 INFO:tasks.rados.rados.0.smithi027.stdout:160: finishing rollback tid 2 to smithi02732947-31 2024-09-15T21:00:48.463 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 31 v 44 (ObjNum 73 snap 8 seq_num 73) dirty exists 2024-09-15T21:00:48.464 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:48.460+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:48.484 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.484 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:155: done (6 left) 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:157: done (5 left) 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:158: done (4 left) 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:160: done (3 left) 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:161: done (2 left) 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 3 current snap is 13 2024-09-15T21:00:48.535 INFO:tasks.rados.rados.0.smithi027.stdout:162: seq_num 81 ranges {0=606208,606208=720896,1327104=49152} 2024-09-15T21:00:48.538 INFO:tasks.rados.rados.0.smithi027.stdout:162: writing smithi02732947-3 from 0 to 606208 tid 1 2024-09-15T21:00:48.543 INFO:tasks.rados.rados.0.smithi027.stdout:162: writing smithi02732947-3 from 606208 to 1327104 tid 2 2024-09-15T21:00:48.543 INFO:tasks.rados.rados.0.smithi027.stdout:162: writing smithi02732947-3 from 1327104 to 1376256 tid 3 2024-09-15T21:00:48.543 INFO:tasks.rados.rados.0.smithi027.stdout:163: copy_from oid 8 from oid 22 current snap is 13 2024-09-15T21:00:48.543 INFO:tasks.rados.rados.0.smithi027.stdout:164: snap_create 2024-09-15T21:00:48.565 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:48.559+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:48.569 INFO:tasks.rados.rados.0.smithi027.stdout:156: finishing write tid 4 to smithi02732947-10 2024-09-15T21:00:48.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:48.567+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:48.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:48.567+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:48.631 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.631 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:00:48.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:48.674 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:48.667+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:48.691 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:48.689+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:48.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:48.731+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:00:48.753 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.749+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:48.754 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.749+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:48.754 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.749+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3 to read 0,1,4,5 2024-09-15T21:00:48.767 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.764+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:48.827 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.827 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.838 INFO:tasks.rados.rados.0.smithi027.stdout:156: finishing write tid 5 to smithi02732947-10 2024-09-15T21:00:48.838 INFO:tasks.rados.rados.0.smithi027.stdout:156: oid 10 updating version 0 to 41 2024-09-15T21:00:48.838 INFO:tasks.rados.rados.0.smithi027.stdout:156: oid 10 version 41 is already newer than 40 2024-09-15T21:00:48.838 INFO:tasks.rados.rados.0.smithi027.stdout:156: oid 10 updating version 41 to 42 2024-09-15T21:00:48.838 INFO:tasks.rados.rados.0.smithi027.stdout:156: oid 10 version 42 is already newer than 39 2024-09-15T21:00:48.839 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 10 v 42 (ObjNum 80 snap 12 seq_num 80) dirty exists 2024-09-15T21:00:48.839 INFO:tasks.rados.rados.0.smithi027.stdout:156: left oid 10 (ObjNum 80 snap 12 seq_num 80) 2024-09-15T21:00:48.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:48.853 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.848+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:48.854 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.848+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:48.855 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.853+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:00:48.855 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.852+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5 to read 0,1,4,5 2024-09-15T21:00:48.868 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:48.864+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:48.957 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.952+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:48.957 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.952+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:48.957 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.952+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:48.957 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:48.952+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:00:48.964 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:48.964 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:48.969 INFO:tasks.rados.rados.0.smithi027.stdout:163: finishing copy_from racing read to smithi02732947-8 2024-09-15T21:00:48.969 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:48.963+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:48.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:49.008 INFO:tasks.rados.rados.0.smithi027.stdout:162: finishing write tid 1 to smithi02732947-3 2024-09-15T21:00:49.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:49.067+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:49.132 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:49.132 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:49.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:49.158 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:49.156+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:49.158 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:49.156+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:49.159 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:49.156+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:49.171 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:49.168+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:49.208 INFO:tasks.rados.rados.0.smithi027.stdout:162: finishing write tid 2 to smithi02732947-3 2024-09-15T21:00:49.209 INFO:tasks.rados.rados.0.smithi027.stdout:162: finishing write tid 3 to smithi02732947-3 2024-09-15T21:00:49.245 INFO:tasks.rados.rados.0.smithi027.stdout:162: finishing write tid 4 to smithi02732947-3 2024-09-15T21:00:49.247 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.244+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:49.247 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.244+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:49.247 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.244+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:00:49.247 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.244+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:00:49.259 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:49.256+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:00:49.260 INFO:tasks.rados.rados.0.smithi027.stdout:159: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-15T21:00:49.262 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 206723 2024-09-15T21:00:49.263 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:49.269 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 175223 2024-09-15T21:00:49.272 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:49.268+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:49.274 INFO:tasks.rados.rados.0.smithi027.stdout:163: finishing copy_from to smithi02732947-8 2024-09-15T21:00:49.274 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 35 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T21:00:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:49.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:49.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:162: finishing write tid 5 to smithi02732947-3 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:162: oid 3 updating version 0 to 38 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:162: oid 3 version 38 is already newer than 36 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:162: oid 3 version 38 is already newer than 37 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:162: oid 3 version 38 is already newer than 35 2024-09-15T21:00:49.348 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 3 v 38 (ObjNum 81 snap 13 seq_num 81) dirty exists 2024-09-15T21:00:49.349 INFO:tasks.rados.rados.0.smithi027.stdout:162: left oid 3 (ObjNum 81 snap 13 seq_num 81) 2024-09-15T21:00:49.372 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:49.368+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:49.467 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:49.468 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:49.473 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:49.467+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:49.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:49.499 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:49.516 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:49.516 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:00:49.534 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:49.534 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:49.540 INFO:tasks.rados.rados.0.smithi027.stdout:156: done (4 left) 2024-09-15T21:00:49.540 INFO:tasks.rados.rados.0.smithi027.stdout:159: done (3 left) 2024-09-15T21:00:49.541 INFO:tasks.rados.rados.0.smithi027.stdout:162: done (2 left) 2024-09-15T21:00:49.541 INFO:tasks.rados.rados.0.smithi027.stdout:163: done (1 left) 2024-09-15T21:00:49.541 INFO:tasks.rados.rados.0.smithi027.stdout:164: done (0 left) 2024-09-15T21:00:49.541 INFO:tasks.rados.rados.0.smithi027.stdout:165: delete oid 31 current snap is 14 2024-09-15T21:00:49.554 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:49.554 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:49.574 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:49.572+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:49.580 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:49.601 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:49.602 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:49.609 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.631626430000001, 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:00:49.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:00:49.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.629615184999999, 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.618 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:00:49.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.628863531, 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:00:49.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:00:49.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:00:49.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.628797346999999, 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.628 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:00:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:00:49.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:129 3.7s0 3:e8b6cabe:::smithi02732947-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.329692061999999, 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61157217500000005, 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 129 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.634 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278370+0000", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6129999999999998e-06 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278383+0000", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3026e-05 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278414+0000", 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0857e-05 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278434+0000", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9924000000000001e-05 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366928+0000", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088493789000000003 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367150+0000", 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000222081 2024-09-15T21:00:49.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690379+0000", 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32322944100000001 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690606+0000", 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022645599999999999 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.889940+0000", 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19933398799999999 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.123536521999998, 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:00:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.122099572, 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:00:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:00:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.122021427, 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.651 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:00:49.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:00:49.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 25.019506293999999, 2024-09-15T21:00:49.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.939459567, 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:00:49.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.925071476999999, 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:00:49.664 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:00:49.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:00:49.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.918661522000001, 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:00:49.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:00:49.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:00:49.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:49.675 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.820013758999998, 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60734763000000003, 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:49.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788048+0000", 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.063e-06 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788069+0000", 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1321000000000001e-05 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788097+0000", 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8136999999999998e-05 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888193+0000", 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100095434 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394534+0000", 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50634117999999995 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395394+0000", 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00085949500000000005 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.818108549000002, 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:00:49.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:00:49.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:00:49.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:49.684 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.736151172, 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62454644199999998, 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871910+0000", 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.55e-06 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871958+0000", 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7698000000000003e-05 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871989+0000", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1061e-05 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.872007+0000", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7934000000000001e-05 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496399+0000", 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62439274499999997 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496455+0000", 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5454000000000001e-05 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.719283732000001, 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60786671999999997, 2024-09-15T21:00:49.688 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888777+0000", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4e-07 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888795+0000", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8403999999999999e-05 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888808+0000", 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2359e-05 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888816+0000", 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3359999999999995e-06 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496583+0000", 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60776654399999996 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496643+0000", 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0136999999999997e-05 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:160 3.2s0 3:48a96439:::smithi02732947-3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e33)", 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.315043430999999, 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61085863699999998, 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 160 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293018+0000", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.074e-06 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293027+0000", 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3820000000000004e-06 2024-09-15T21:00:49.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293049+0000", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2697999999999999e-05 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293072+0000", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671e-05 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699705+0000", 2024-09-15T21:00:49.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406633417 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699975+0000", 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026989499999999998 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.701228+0000", 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012527560000000001 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.702476+0000", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001248033 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903693+0000", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20121672600000001 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903875+0000", 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018198500000000001 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.697 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.4074849599999997, 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:49.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:00:49.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:00:49.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:00:49.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.3045310099999998, 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66775810800000002, 2024-09-15T21:00:49.703 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303530+0000", 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8299999999999995e-07 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303918+0000", 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038879600000000001 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303931+0000", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-05 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303938+0000", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4359999999999999e-06 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971275+0000", 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66733690499999998 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971287+0000", 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2007999999999999e-05 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.2973044250000001, 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66056677200000002, 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:49.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310756+0000", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311135+0000", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037949099999999998 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311146+0000", 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0736e-05 2024-09-15T21:00:49.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311151+0000", 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3299999999999998e-06 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971311+0000", 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66015986699999996 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:49.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971322+0000", 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:49.711 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:49.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:49.713 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:49.714 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:49.714 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:49.715 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:49.715 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:49.716 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:49.708+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:49.733 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:49.774 INFO:tasks.rados.rados.0.smithi027.stdout:165: done (0 left) 2024-09-15T21:00:49.774 INFO:tasks.rados.rados.0.smithi027.stdout:166: delete oid 26 current snap is 14 2024-09-15T21:00:49.813 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.808+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 58/50 ECSubWrite(tid=140, reqid=client.4641.0:424, at_version=0'0, trim_to=0'0, roll_forward_to=58'46))", 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:49.773886+0000", 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.079028876999999997, 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.079083210000000001, 2024-09-15T21:00:49.854 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773886+0000", 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773886+0000", 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:49.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773888+0000", 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2859999999999998e-06 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773920+0000", 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.129e-05 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:49.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773945+0000", 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5517e-05 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:49.773960+0000", 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4255e-05 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:00:49.857 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:49.868 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:49.915 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:49.912+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:49.989 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:49.990 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:49.990 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:49.990 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:49.990 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:50.005 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:50.051 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:50.047+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:50.117 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:50.112+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.361494647000001, 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:00:50.135 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:00:50.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:00:50.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.360393281, 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:00:50.141 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:00:50.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.360352776999999, 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:50.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:00:50.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:00:50.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:00:50.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.339801267999999, 2024-09-15T21:00:50.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:00:50.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:00:50.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.271513670000001, 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:00:50.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:00:50.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:00:50.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.238494412000001, 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.166 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:00:50.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:00:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:00:50.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.238454021999999, 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.172 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:00:50.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.170241857000001, 2024-09-15T21:00:50.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:00:50.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.168339052, 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:00:50.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:00:50.184 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.166280723, 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202146044, 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965234+0000", 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7099999999999999e-06 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965259+0000", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5732999999999999e-05 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965292+0000", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2223e-05 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965310+0000", 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8087000000000001e-05 2024-09-15T21:00:50.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167355+0000", 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202045846 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167378+0000", 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2444999999999999e-05 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.156356984999999, 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:50.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:00:50.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.193 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.155547245000001, 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:00:50.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.055615541000002, 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.035939509999999, 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:00:50.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.204 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:00:50.205 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=6, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.751831645999999, 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19803510199999999, 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.206 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379683+0000", 2024-09-15T21:00:50.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.917e-06 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379693+0000", 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.01e-05 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379722+0000", 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8623e-05 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379740+0000", 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8677000000000001e-05 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577690+0000", 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19795005800000001 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577716+0000", 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5726999999999999e-05 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:50.210 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.744497795000001, 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.211 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:00:50.212 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.213 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.214 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.547074728999998, 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:50.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:00:50.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=18, reqid=client.4641.0:143, at_version=33'4, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:50.219 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.542689428999999, 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013446600000001, 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.220 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588825+0000", 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.221 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588950+0000", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124919 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588978+0000", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8500000000000002e-05 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588993+0000", 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5237e-05 2024-09-15T21:00:50.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785673+0000", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19668001800000001 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785700+0000", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7112999999999999e-05 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788921+0000", 2024-09-15T21:00:50.223 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032205490000000001 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788958+0000", 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6619999999999998e-05 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:50.224 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.463838921000001, 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.225 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.226 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:00:50.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.228 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.771642452, 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.229 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:00:50.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.232 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.233 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.233 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:50.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:50.235 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:50.227+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:50.255 INFO:tasks.rados.rados.0.smithi027.stdout:166: done (0 left) 2024-09-15T21:00:50.255 INFO:tasks.rados.rados.0.smithi027.stdout:167: copy_from oid 20 from oid 42 current snap is 14 2024-09-15T21:00:50.255 INFO:tasks.rados.rados.0.smithi027.stdout:168: delete oid 5 current snap is 14 2024-09-15T21:00:50.335 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:50.332+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.0.17:2 3.5s0 3:a50cee1c:::smithi02732947-42:head [assert-version v70,copy-get max 8388608 in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.356920+0000", 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.061612841000000002, 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.061664287999999998, 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.420 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356920+0000", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356920+0000", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356923+0000", 2024-09-15T21:00:50.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356936+0000", 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2934e-05 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356958+0000", 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2946e-05 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.356977+0000", 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8223000000000002e-05 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.357072+0000", 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5435999999999999e-05 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.357180+0000", 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000108322 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.423 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 58/50 ECSubReadReply(tid=178, attrs_read=0))", 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.358651+0000", 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.059882544000000003, 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.060023146999999999, 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358651+0000", 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358651+0000", 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358652+0000", 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.235e-06 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359444+0000", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00079185900000000001 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359457+0000", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3213999999999999e-05 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359467+0000", 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7920000000000004e-06 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 58/50 ECSubReadReply(tid=178, attrs_read=0))", 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.359431+0000", 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.059102374999999999, 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.059306781000000003, 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359431+0000", 2024-09-15T21:00:50.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359431+0000", 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359431+0000", 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359431+0000", 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359475+0000", 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4212999999999998e-05 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359493+0000", 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8233e-05 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.429 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 58/50 ECSubReadReply(tid=178, attrs_read=0))", 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.358730+0000", 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.059802907000000002, 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.060067579000000003, 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358730+0000", 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358730+0000", 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.358732+0000", 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9700000000000002e-06 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359625+0000", 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089272400000000001 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359656+0000", 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1032999999999999e-05 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359676+0000", 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9944000000000001e-05 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 58/50 ECSubReadReply(tid=178, attrs_read=0))", 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.359532+0000", 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.059000876000000001, 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.059326447999999997, 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.433 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359532+0000", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359532+0000", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.434 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359533+0000", 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9599999999999999e-07 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359808+0000", 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027555899999999998 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.359824+0000", 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5906e-05 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 58/50 ECSubReadReply(tid=178, attrs_read=0))", 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.360445+0000", 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.058088251, 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.058470199, 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.360445+0000", 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.360445+0000", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.360446+0000", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.159e-06 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.361188+0000", 2024-09-15T21:00:50.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074178899999999997 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.361213+0000", 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4978999999999999e-05 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.361226+0000", 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2764e-05 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.438 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.439 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:50.439 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 6 2024-09-15T21:00:50.439 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:50.440 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:50.441 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:50.436+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:50.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:50.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:50.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:50.455+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:00:50.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:50.455+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:00:50.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:50.455+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:50.477 INFO:tasks.rados.rados.0.smithi027.stdout:167: finishing copy_from racing read to smithi02732947-20 2024-09-15T21:00:50.541 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:50.536+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:50.562 INFO:tasks.rados.rados.0.smithi027.stdout:168: done (1 left) 2024-09-15T21:00:50.562 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 13 current snap is 14 2024-09-15T21:00:50.562 INFO:tasks.rados.rados.0.smithi027.stdout:169: seq_num 85 ranges {1376256=671744,2048000=770048} 2024-09-15T21:00:50.563 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:50.564 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:50.583 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:50.597 INFO:tasks.rados.rados.0.smithi027.stdout:169: writing smithi02732947-13 from 1376256 to 2048000 tid 1 2024-09-15T21:00:50.609 INFO:tasks.rados.rados.0.smithi027.stdout:169: writing smithi02732947-13 from 2048000 to 2818048 tid 2 2024-09-15T21:00:50.609 INFO:tasks.rados.rados.0.smithi027.stdout:170: read oid 19 snap -1 2024-09-15T21:00:50.609 INFO:tasks.rados.rados.0.smithi027.stdout:170: expect deleted 2024-09-15T21:00:50.609 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 15 current snap is 14 2024-09-15T21:00:50.610 INFO:tasks.rados.rados.0.smithi027.stdout:171: seq_num 86 ranges {229376=688128,917504=540672,1458176=229376} 2024-09-15T21:00:50.622 INFO:tasks.rados.rados.0.smithi027.stdout:171: writing smithi02732947-15 from 229376 to 917504 tid 1 2024-09-15T21:00:50.629 INFO:tasks.rados.rados.0.smithi027.stdout:171: writing smithi02732947-15 from 917504 to 1458176 tid 2 2024-09-15T21:00:50.631 INFO:tasks.rados.rados.0.smithi027.stdout:171: writing smithi02732947-15 from 1458176 to 1687552 tid 3 2024-09-15T21:00:50.631 INFO:tasks.rados.rados.0.smithi027.stdout:172: read oid 23 snap -1 2024-09-15T21:00:50.631 INFO:tasks.rados.rados.0.smithi027.stdout:172: expect (ObjNum 70 snap 7 seq_num 70) 2024-09-15T21:00:50.631 INFO:tasks.rados.rados.0.smithi027.stdout:173: snap_remove snap 13 2024-09-15T21:00:50.683 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:50.667+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:50.743 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:50.740+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:50.756 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:121 3.5s0 3:ac8a7abe:::smithi02732947-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.592832331, 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82723362600000006, 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.757 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 121 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.758 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161396+0000", 2024-09-15T21:00:50.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3630000000000001e-06 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162612+0000", 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001215556 2024-09-15T21:00:50.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.763 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162641+0000", 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.904e-05 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162654+0000", 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.33e-05 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262983+0000", 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100329088 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263223+0000", 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023989200000000001 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265887+0000", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002663978 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369669+0000", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10378166899999999 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988509+0000", 2024-09-15T21:00:50.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61884030999999995 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988628+0000", 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011843 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.767 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:122 3.5s0 3:ac8a7abe:::smithi02732947-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.591542216000001, 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82610852099999998, 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 122 2024-09-15T21:00:50.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.769 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162685+0000", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2500000000000001e-06 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162691+0000", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2900000000000002e-06 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162703+0000", 2024-09-15T21:00:50.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1902999999999999e-05 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162711+0000", 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3000000000000002e-06 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473128+0000", 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31041744100000002 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473278+0000", 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014961900000000001 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.772 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474169+0000", 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089135400000000002 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587329+0000", 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113159912 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988729+0000", 2024-09-15T21:00:50.773 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40140014600000001 2024-09-15T21:00:50.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988793+0000", 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3305999999999993e-05 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.776 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.591469913000001, 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.777 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:50.778 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.779 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.780 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:00:50.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.782 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.783 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.591427546999999, 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:00:50.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.785 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:00:50.786 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.787 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.788 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:00:50.789 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.790 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.591036620000001, 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.791 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:00:50.792 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.793 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.794 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:00:50.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.796 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.590997343000002, 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.797 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:00:50.800 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.802 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:50.803 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.551765587999999, 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.804 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.805 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:00:50.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.807 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:00:50.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.810 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.810 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.551043236000002, 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.811 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.812 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.813 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.814 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.814 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:00:50.814 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.816 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.551026745000001, 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.817 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:00:50.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:00:50.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:00:50.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.057839385000001, 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:00:50.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:00:50.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:00:50.826 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.051397921, 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:00:50.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.828 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.829 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:50.832 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.050971519000001, 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:50.833 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:00:50.834 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:00:50.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:238 3.5s0 3:a50cee1c:::smithi02732947-42:head [copy-from ver 10 in=55b] snapc 4=[4,3] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.680321770999999, 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.711200465, 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 238 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.838 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073908+0000", 2024-09-15T21:00:50.839 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1200000000000002e-06 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073922+0000", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4392999999999999e-05 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073954+0000", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1439000000000002e-05 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073974+0000", 2024-09-15T21:00:50.840 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0922999999999999e-05 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.074945+0000", 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097046399999999998 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.075186+0000", 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024103199999999999 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.841 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.281399+0000", 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20621348 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.483210+0000", 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20181083799999999 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.784991+0000", 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301780257 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.842 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.785105+0000", 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011451900000000001 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:252 3.cs0 3:3f0b4dcc:::smithi02732947-9:head [setxattr _header (55) in=7b] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.124704+0000", 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.629522646000002, 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.65147824600000004, 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:50.843 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 252 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124704+0000", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124704+0000", 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124705+0000", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1739999999999999e-06 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124710+0000", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3800000000000002e-06 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124725+0000", 2024-09-15T21:00:50.845 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4253000000000001e-05 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124735+0000", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.029e-05 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.275021+0000", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15028593800000001 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.275270+0000", 2024-09-15T21:00:50.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024921200000000001 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.276557+0000", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012873979999999999 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.277649+0000", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001091498 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776062+0000", 2024-09-15T21:00:50.847 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.498413314 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776182+0000", 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119789 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:253 3.cs0 3:3f0b4dcc:::smithi02732947-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.629476162, 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.65441706499999996, 2024-09-15T21:00:50.848 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 253 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.849 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124751+0000", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124755+0000", 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5499999999999999e-06 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.850 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124767+0000", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2758999999999999e-05 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124779+0000", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1209999999999999e-05 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.277194+0000", 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15241518800000001 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.277449+0000", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025487099999999999 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776732+0000", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49928296 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776836+0000", 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010461799999999999 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.779167+0000", 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023311289999999999 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:349 3.5s0 3:a5da2594:::smithi02732947-25:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.612030024999999, 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.724849717, 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 349 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:00:50.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142198+0000", 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4559999999999999e-06 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142206+0000", 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9130000000000003e-06 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142220+0000", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4253999999999999e-05 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142230+0000", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0423e-05 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142468+0000", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000237712 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:50.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142491+0000", 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2476e-05 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.456194+0000", 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31370373499999998 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.456355+0000", 2024-09-15T21:00:50.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000160896 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.867046+0000", 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.410690852 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:350 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:50.858 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.61189261, 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82820832200000005, 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 350 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:00:50.859 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142335+0000", 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.717e-06 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.860 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142346+0000", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0869999999999999e-05 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142374+0000", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7322999999999999e-05 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142395+0000", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1063000000000001e-05 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142587+0000", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019266899999999999 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142604+0000", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6532999999999999e-05 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459842+0000", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31723833800000001 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.862 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459989+0000", 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014666899999999999 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.970542+0000", 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51055313999999996 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:351 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.611778902999999, 2024-09-15T21:00:50.863 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.83058971800000003, 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 351 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142449+0000", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.663e-06 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142459+0000", 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0083e-05 2024-09-15T21:00:50.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142477+0000", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7842999999999998e-05 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142492+0000", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-05 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242885+0000", 2024-09-15T21:00:50.866 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100392596 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242896+0000", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1382999999999999e-05 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460558+0000", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217662156 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460682+0000", 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124256 2024-09-15T21:00:50.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.973037+0000", 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51235473799999998 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.611664280999999, 2024-09-15T21:00:50.868 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:00:50.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.873 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.5s0 48/28 ECSubReadReply(tid=134, attrs_read=0))", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.660917+0000", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.093309106, 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61591166100000005, 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.660917+0000", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:50.874 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.660917+0000", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.660918+0000", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5400000000000002e-07 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.661201+0000", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028377099999999999 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:50.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.661224+0000", 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2643e-05 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.761335+0000", 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011070399999999 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.276802+0000", 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51546704200000004 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:50.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.276829+0000", 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7047e-05 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:50.877 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:50.878 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.860+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:50.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:50.879 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:50.875+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:50.964 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:00:50.964 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:50.964 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:50.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4 to read 0,1,4,5 2024-09-15T21:00:50.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:50.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,4,5 2024-09-15T21:00:50.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.960+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:50.974 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.972+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:00:50.974 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.972+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:00:50.977 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.972+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3 to read 0,1,4,5 2024-09-15T21:00:50.979 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:50.976+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:50.981 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:50.976+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,4 to read 0,1,4,5 2024-09-15T21:00:51.057 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.18792311, 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.18796373, 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:00:51.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.061 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 59/57 ECSubRead(tid=7, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.968776+0000", 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.087707997999999995, 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.087851671000000006, 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968776+0000", 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968776+0000", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968777+0000", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.265e-06 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968789+0000", 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2493999999999999e-05 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968806+0000", 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6147999999999999e-05 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968816+0000", 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0217000000000001e-05 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.064 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.968827+0000", 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.087656369999999997, 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.087873258999999995, 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968827+0000", 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968827+0000", 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968828+0000", 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968836+0000", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1280000000000008e-06 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968843+0000", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.759e-06 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968849+0000", 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0100000000000001e-06 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.067 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.087627024999999997, 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.087930465999999999, 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8299999999999998e-07 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968860+0000", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968883+0000", 2024-09-15T21:00:51.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2989999999999998e-05 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968894+0000", 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0553e-05 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:00:51.071 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 4 2024-09-15T21:00:51.072 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:51.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:51.081 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:51.076+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:51.181 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:51.176+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:51.194 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.194 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:51.194 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:51.194 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:51.194 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:51.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:51.283 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:51.280+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.333 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.049455227999999, 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20052589700000001, 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280374+0000", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280375+0000", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.567e-06 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280390+0000", 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4800000000000001e-05 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280411+0000", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1009000000000002e-05 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280423+0000", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1717e-05 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480839+0000", 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20041608799999999 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480900+0000", 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0715999999999999e-05 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.049378924999999, 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:00:51.337 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:00:51.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.047648432999999, 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.341 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:00:51.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:00:51.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:00:51.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.046400855000002, 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.345 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.348 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.032516750999999, 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:00:51.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.as0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.948715305, 2024-09-15T21:00:51.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20016901200000001, 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381114+0000", 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381115+0000", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4559999999999999e-06 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381122+0000", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381146+0000", 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3731999999999999e-05 2024-09-15T21:00:51.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381155+0000", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0340000000000002e-06 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581244+0000", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20008901500000001 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581283+0000", 2024-09-15T21:00:51.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8884000000000002e-05 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.948634751, 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.359 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.846187802999999, 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:00:51.364 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.845268905000001, 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.368 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.842745636, 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.369 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:00:51.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:00:51.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:00:51.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.375 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.840448598000002, 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.379 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:00:51.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.750851821000001, 2024-09-15T21:00:51.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:00:51.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:00:51.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.747939141, 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.386 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:00:51.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 38/28 ECSubWrite(tid=72, reqid=osd.5.0:72, at_version=38'32, trim_to=0'0, roll_forward_to=37'28))", 2024-09-15T21:00:51.390 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.151430013999999, 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20162162, 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:51.391 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178399+0000", 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0100000000000005e-07 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178420+0000", 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0568e-05 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178431+0000", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0947e-05 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.178455+0000", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3533e-05 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.378594+0000", 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013990700000001 2024-09-15T21:00:51.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.378603+0000", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2819999999999996e-06 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.379987+0000", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013840250000000001 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.380021+0000", 2024-09-15T21:00:51.394 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3856999999999998e-05 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs7 39/28 ECSubWrite(tid=74, reqid=client.4641.0:270, at_version=39'13, trim_to=0'0, roll_forward_to=33'9))", 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.656061+0000", 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.67376805, 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201302594, 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.656061+0000", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.656061+0000", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.656063+0000", 2024-09-15T21:00:51.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.637e-06 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.656103+0000", 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0833e-05 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.656140+0000", 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6455999999999998e-05 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.756230+0000", 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10009053 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.856459+0000", 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10022874399999999 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.856490+0000", 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1056000000000002e-05 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.857330+0000", 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083971700000000002 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.857364+0000", 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3621000000000002e-05 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.399 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 10.866389858, 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:00:51.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:00:51.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:00:51.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/28 ECSubWrite(tid=48, reqid=client.4641.0:387, at_version=53'12, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.680156588, 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203297113, 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649673+0000", 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1199999999999999e-07 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649825+0000", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015242899999999999 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649841+0000", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5931e-05 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.749979+0000", 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100138358 2024-09-15T21:00:51.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750181+0000", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020151199999999999 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750213+0000", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1763000000000001e-05 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852931+0000", 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102718235 2024-09-15T21:00:51.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852970+0000", 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8572999999999998e-05 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/28 ECSubWrite(tid=51, reqid=client.4641.0:390, at_version=53'15, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.857991+0000", 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.4718379559999999, 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201440006, 2024-09-15T21:00:51.408 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.857991+0000", 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.857991+0000", 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.857994+0000", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5900000000000002e-06 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.858055+0000", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1669999999999997e-05 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.858085+0000", 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9468999999999999e-05 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.958175+0000", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100090264 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:45.058375+0000", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100199723 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:45.058390+0000", 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5637e-05 2024-09-15T21:00:51.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:45.059353+0000", 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00096305000000000002 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:45.059431+0000", 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7602999999999995e-05 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.412 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.46126845700000002, 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:00:51.416 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.36097237199999999, 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20316889599999999, 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8299999999999998e-07 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968860+0000", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968883+0000", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2989999999999998e-05 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968894+0000", 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0553e-05 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.171727+0000", 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20283357299999999 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172026+0000", 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00029805000000000001 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.420 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:51.422 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:51.388+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:51.422 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:51.388+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:00:51.422 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:51.388+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,4,5 2024-09-15T21:00:51.422 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:51.388+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:00:51.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:51.423 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:51.416+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:51.486 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.479+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:51.486 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.479+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:00:51.486 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.479+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,7 to read 0,1,4,5 2024-09-15T21:00:51.486 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.479+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6,7 to read 0,1,4,5 2024-09-15T21:00:51.523 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:51.520+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:51.556 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.556 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:51.556 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:51.556 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:51.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:00:51.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:51.573 INFO:tasks.rados.rados.0.smithi027.stdout:173: done (5 left) 2024-09-15T21:00:51.573 INFO:tasks.rados.rados.0.smithi027.stdout:174: read oid 8 snap 9 2024-09-15T21:00:51.573 INFO:tasks.rados.rados.0.smithi027.stdout:174: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-15T21:00:51.573 INFO:tasks.rados.rados.0.smithi027.stdout:175: copy_from oid 24 from oid 46 current snap is 14 2024-09-15T21:00:51.573 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 41 current snap is 14 2024-09-15T21:00:51.574 INFO:tasks.rados.rados.0.smithi027.stdout:176: seq_num 87 ranges {1785856=704512,2490368=688128,3178496=81920} 2024-09-15T21:00:51.584 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.576+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:51.584 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.576+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:00:51.588 INFO:tasks.rados.rados.0.smithi027.stdout:176: writing smithi02732947-41 from 1785856 to 2490368 tid 1 2024-09-15T21:00:51.594 INFO:tasks.rados.rados.0.smithi027.stdout:176: writing smithi02732947-41 from 2490368 to 3178496 tid 2 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:176: writing smithi02732947-41 from 3178496 to 3260416 tid 3 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:167: finishing copy_from to smithi02732947-20 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 20 v 70 (ObjNum 77 snap 9 seq_num 77) dirty exists 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:167: done (7 left) 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:177: copy_from oid 47 from oid 7 current snap is 14 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:171: finishing write tid 1 to smithi02732947-15 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:172: expect (ObjNum 70 snap 7 seq_num 70) 2024-09-15T21:00:51.595 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:51.624 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.619+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:51.657 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 420736 2024-09-15T21:00:51.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.672+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:00:51.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.676+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:51.687 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:51.700 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.701 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:51.701 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:51.701 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:51.701 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:51.715 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: finishing write tid 1 to smithi02732947-13 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:171: finishing write tid 2 to smithi02732947-15 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:171: finishing write tid 3 to smithi02732947-15 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: finishing write tid 2 to smithi02732947-13 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: finishing write tid 3 to smithi02732947-13 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: finishing write tid 4 to smithi02732947-13 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: oid 13 updating version 0 to 76 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: oid 13 version 76 is already newer than 75 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: oid 13 version 76 is already newer than 74 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 76 (ObjNum 85 snap 14 seq_num 85) dirty exists 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:169: left oid 13 (ObjNum 85 snap 14 seq_num 85) 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:171: finishing write tid 4 to smithi02732947-15 2024-09-15T21:00:51.717 INFO:tasks.rados.rados.0.smithi027.stdout:171: finishing write tid 5 to smithi02732947-15 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: oid 15 updating version 0 to 12 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: oid 15 updating version 12 to 13 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: oid 15 version 13 is already newer than 11 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: oid 15 updating version 13 to 14 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 15 v 14 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: left oid 15 (ObjNum 86 snap 14 seq_num 86) 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:169: done (7 left) 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:170: done (6 left) 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:171: done (5 left) 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:172: done (4 left) 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:178: rollback oid 25 current snap is 14 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 25 to 12 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 13 current snap is 14 2024-09-15T21:00:51.718 INFO:tasks.rados.rados.0.smithi027.stdout:179: seq_num 88 ranges {2818048=720896,3538944=442368,3981312=327680} 2024-09-15T21:00:51.736 INFO:tasks.rados.rados.0.smithi027.stdout:179: writing smithi02732947-13 from 2818048 to 3538944 tid 1 2024-09-15T21:00:51.738 INFO:tasks.rados.rados.0.smithi027.stdout:179: writing smithi02732947-13 from 3538944 to 3981312 tid 2 2024-09-15T21:00:51.740 INFO:tasks.rados.rados.0.smithi027.stdout:179: writing smithi02732947-13 from 3981312 to 4308992 tid 3 2024-09-15T21:00:51.741 INFO:tasks.rados.rados.0.smithi027.stdout:180: rollback oid 22 current snap is 14 2024-09-15T21:00:51.741 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 22 to 12 2024-09-15T21:00:51.741 INFO:tasks.rados.rados.0.smithi027.stdout:181: snap_remove snap 11 2024-09-15T21:00:51.744 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:51.744 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:51.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:00:51.754 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:51.759 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:51.755+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:51.782 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.780+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:00:51.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:51.784+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:51.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:51.784+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:51.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:51.784+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:00:51.791 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:51.784+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,7 to read 0,1,4,5 2024-09-15T21:00:51.826 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:51.824+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:51.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.880+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:51.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.880+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:51.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.880+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:51.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:51.880+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6 to read 0,1,4,5 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:51.890 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.267018853000003, 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:51.891 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:00:51.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.895 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.896 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:51.897 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:51.897 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.653724594000003, 2024-09-15T21:00:51.897 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.898 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:00:51.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.903 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.653539891999998, 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:00:51.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:00:51.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.448263881000003, 2024-09-15T21:00:51.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:51.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:00:51.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.916 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.447938720000003, 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:00:51.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:00:51.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:00:51.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.447897863000001, 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:51.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.447849116999997, 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.930 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:51.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:00:51.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:00:51.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.936 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.447534206, 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.937 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:00:51.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.942 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.44745614, 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:00:51.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.948 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.657000435, 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:00:51.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.656830392000003, 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:00:51.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.960 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:437 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~540672] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.256411071, 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54912561999999998, 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:51.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 437 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628861+0000", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7499999999999999e-06 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629602+0000", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142900000000005 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629633+0000", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0453000000000001e-05 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629649+0000", 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5923e-05 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629768+0000", 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119453 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.630003+0000", 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023483799999999999 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.632434+0000", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024312940000000001 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.636553+0000", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041183879999999997 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177838+0000", 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54128565900000003 2024-09-15T21:00:51.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177984+0000", 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014543299999999999 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:438 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~229376] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.2538423750000001, 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54706032999999998, 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 438 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631429+0000", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2630000000000002e-06 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631860+0000", 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000431261 2024-09-15T21:00:51.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631886+0000", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6230000000000001e-05 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631907+0000", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633475+0000", 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015675870000000001 2024-09-15T21:00:51.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633636+0000", 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016115899999999999 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.635458+0000", 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018221299999999999 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.637212+0000", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017535420000000001 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178395+0000", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54118301599999996 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178487+0000", 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2262999999999994e-05 2024-09-15T21:00:51.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:439 3.ds0 3:b220672c:::smithi02732947-15:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.253315875, 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.85063377100000004, 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 439 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631956+0000", 2024-09-15T21:00:51.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-06 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631966+0000", 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0047000000000001e-05 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631987+0000", 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1253000000000001e-05 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732106+0000", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011937999999999 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179003+0000", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44689677 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179116+0000", 2024-09-15T21:00:51.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112794 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179656+0000", 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054039500000000005 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180543+0000", 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088661899999999995 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482437+0000", 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30189397699999998 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482587+0000", 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015026300000000001 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:440 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:51.979 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.1531189529999999, 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.75401734600000003, 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 440 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732153+0000", 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.717e-06 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732167+0000", 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.382e-05 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732202+0000", 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5553000000000003e-05 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732225+0000", 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2894000000000001e-05 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180004+0000", 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44777894899999998 2024-09-15T21:00:51.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180092+0000", 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7705000000000001e-05 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483149+0000", 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30305710299999999 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483259+0000", 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109998 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.486168+0000", 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0029086070000000001 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:00:51.989 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.1082242259999999, 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40123385299999997, 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3900000000000001e-07 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777047+0000", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7519999999999999e-06 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777053+0000", 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8749999999999997e-06 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777057+0000", 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5580000000000002e-06 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178267+0000", 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40120943799999997 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178279+0000", 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1990999999999999e-05 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.05475721, 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34768011700000001, 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830513+0000", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.587e-06 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830522+0000", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.303e-06 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830549+0000", 2024-09-15T21:00:51.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6820000000000001e-05 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.930645+0000", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100096688 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178169+0000", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.247523936 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178192+0000", 2024-09-15T21:00:51.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2782999999999999e-05 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=107, last_complete=58'7, committed=1, applied=1))", 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.0521405150000001, 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34488795, 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:51.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:51.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:00:51.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833130+0000", 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833135+0000", 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8000000000000004e-06 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833156+0000", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0703000000000002e-05 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833167+0000", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.058e-05 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177702+0000", 2024-09-15T21:00:52.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34453553599999998 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178017+0000", 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031424099999999999 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.0520837190000001, 2024-09-15T21:00:52.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.345738139, 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833186+0000", 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9400000000000005e-07 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833189+0000", 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4860000000000002e-06 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833202+0000", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2116999999999999e-05 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833212+0000", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0633e-05 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178319+0000", 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34510687200000001 2024-09-15T21:00:52.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178924+0000", 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060443699999999999 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.0518844329999999, 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34485581700000001, 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:52.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5999999999999999e-07 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833388+0000", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.48e-06 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833399+0000", 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1749999999999999e-05 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833406+0000", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3960000000000003e-06 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178221+0000", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34481549900000003 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178241+0000", 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9232000000000002e-05 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.009 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:52.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:52.010 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:52.010 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:52.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:52.011 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:52.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:52.012 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:52.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:52.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:52.007+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:52.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:00:52.069 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:00:52.087 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-15T21:00:51.570349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":52773427,"num_objects":64,"num_object_clones":14,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":3,"num_read":292,"num_read_kb":25488,"num_write":497,"num_write_kb":60984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":15499264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":493,"ondisk_log_size":493,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":451988,"kb_used_data":115164,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1132010092,"statfs":{"total":1159641169920,"available":1159178334208,"internally_reserved":0,"allocated":117927936,"data_stored":133710162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,1,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9519104,"num_objects":8,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":-2,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":16,"num_read_kb":2956,"num_write":23,"num_write_kb":9712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":13582336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028095"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":140,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.674207+0000","last_change":"2024-09-15T21:00:47.558849+0000","last_active":"2024-09-15T21:00:50.674207+0000","last_peered":"2024-09-15T21:00:50.674207+0000","last_clean":"2024-09-15T21:00:50.674207+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:00:50.674207+0000","last_undegraded":"2024-09-15T21:00:50.674207+0000","last_fullsized":"2024-09-15T21:00:50.674207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:22.883873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030461189999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"3.e","version":"45'9","reported_seq":146,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:00:47.556417+0000","last_change":"2024-09-15T21:00:47.556805+0000","last_active":"2024-09-15T21:00:47.556417+0000","last_peered":"2024-09-15T21:00:47.556417+0000","last_clean":"2024-09-15T21:00:47.556417+0000","last_became_active":"2024-09-15T21:00:20.214352+0000","last_became_peered":"2024-09-15T21:00:20.214352+0000","last_unstale":"2024-09-15T21:00:47.556417+0000","last_undegraded":"2024-09-15T21:00:47.556417+0000","last_fullsized":"2024-09-15T21:00:47.556417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:42.688035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071546900000000004,"stat_sum":{"num_bytes":802816,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":1257,"num_write":6,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"5"},{"start":"c","length":"1"}]},{"pgid":"3.d","version":"59'9","reported_seq":157,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:51.482441+0000","last_change":"2024-09-15T21:00:47.658998+0000","last_active":"2024-09-15T21:00:51.482441+0000","last_peered":"2024-09-15T21:00:51.482441+0000","last_clean":"2024-09-15T21:00:51.482441+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:00:51.482441+0000","last_undegraded":"2024-09-15T21:00:51.482441+0000","last_fullsized":"2024-09-15T21:00:51.482441+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:42.095071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.102857569,"stat_sum":{"num_bytes":1687552,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1400,"num_write":27,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"3.c","version":"52'37","reported_seq":164,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:00:46.701285+0000","last_change":"2024-09-15T21:00:43.956925+0000","last_active":"2024-09-15T21:00:46.701285+0000","last_peered":"2024-09-15T21:00:46.701285+0000","last_clean":"2024-09-15T21:00:46.701285+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:00:46.701285+0000","last_undegraded":"2024-09-15T21:00:46.701285+0000","last_fullsized":"2024-09-15T21:00:46.701285+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:38.275482+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.40605664899999999,"stat_sum":{"num_bytes":3211264,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":5,"num_write":33,"num_write_kb":3296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"5"}]},{"pgid":"3.b","version":"58'46","reported_seq":201,"reported_epoch":58,"state":"peering","last_fresh":"2024-09-15T21:00:50.189315+0000","last_change":"2024-09-15T21:00:50.570454+0000","last_active":"2024-09-15T21:00:50.570454+0000","last_peered":"2024-09-15T21:00:50.189315+0000","last_clean":"2024-09-15T21:00:41.072625+0000","last_became_active":"2024-09-15T21:00:43.468280+0000","last_became_peered":"2024-09-15T21:00:43.468280+0000","last_unstale":"2024-09-15T21:00:50.189315+0000","last_undegraded":"2024-09-15T21:00:50.189315+0000","last_fullsized":"2024-09-15T21:00:43.150741+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.86869606399999999,"stat_sum":{"num_bytes":5832704,"num_objects":8,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":20,"num_read_kb":4873,"num_write":27,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,4,6,7,8,9,11],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"3.a","version":"32'2","reported_seq":157,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:51.283017+0000","last_change":"2024-09-15T21:00:51.283379+0000","last_active":"2024-09-15T21:00:51.283017+0000","last_peered":"2024-09-15T21:00:51.283017+0000","last_clean":"2024-09-15T21:00:51.283017+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:00:51.283017+0000","last_undegraded":"2024-09-15T21:00:51.283017+0000","last_fullsized":"2024-09-15T21:00:51.283017+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:38.614814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035120379999999999,"stat_sum":{"num_bytes":557056,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"56'35","reported_seq":190,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.675075+0000","last_change":"2024-09-15T21:00:48.210548+0000","last_active":"2024-09-15T21:00:50.675075+0000","last_peered":"2024-09-15T21:00:50.675075+0000","last_clean":"2024-09-15T21:00:50.675075+0000","last_became_active":"2024-09-15T21:00:20.308787+0000","last_became_peered":"2024-09-15T21:00:20.308787+0000","last_unstale":"2024-09-15T21:00:50.675075+0000","last_undegraded":"2024-09-15T21:00:50.675075+0000","last_fullsized":"2024-09-15T21:00:50.675075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:45.623846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.521789909,"stat_sum":{"num_bytes":3506176,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":3259,"num_write":38,"num_write_kb":6000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"3.8","version":"33'3","reported_seq":155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:00:47.555513+0000","last_change":"2024-09-15T21:00:47.555982+0000","last_active":"2024-09-15T21:00:47.555513+0000","last_peered":"2024-09-15T21:00:47.555513+0000","last_clean":"2024-09-15T21:00:47.555513+0000","last_became_active":"2024-09-15T21:00:45.947614+0000","last_became_peered":"2024-09-15T21:00:45.947614+0000","last_unstale":"2024-09-15T21:00:47.555513+0000","last_undegraded":"2024-09-15T21:00:47.555513+0000","last_fullsized":"2024-09-15T21:00:47.555513+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:28.713216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078558499999999999,"stat_sum":{"num_bytes":180224,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":3,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":180224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"5"},{"start":"c","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":203,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.674453+0000","last_change":"2024-09-15T20:59:57.790315+0000","last_active":"2024-09-15T21:00:50.674453+0000","last_peered":"2024-09-15T21:00:50.674453+0000","last_clean":"2024-09-15T21:00:50.674453+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:00:50.674453+0000","last_undegraded":"2024-09-15T21:00:50.674453+0000","last_fullsized":"2024-09-15T21:00:50.674453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T20:59:56.411296+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:03.864701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:00:46.700859+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:00:46.700859+0000","last_peered":"2024-09-15T21:00:46.700859+0000","last_clean":"2024-09-15T21:00:46.700859+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:00:46.700859+0000","last_undegraded":"2024-09-15T21:00:46.700859+0000","last_fullsized":"2024-09-15T21:00:46.700859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"58'40","reported_seq":194,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.567844+0000","last_change":"2024-09-15T21:00:47.577706+0000","last_active":"2024-09-15T21:00:50.567844+0000","last_peered":"2024-09-15T21:00:50.567844+0000","last_clean":"2024-09-15T21:00:50.567844+0000","last_became_active":"2024-09-15T21:00:20.411551+0000","last_became_peered":"2024-09-15T21:00:20.411551+0000","last_unstale":"2024-09-15T21:00:50.567844+0000","last_undegraded":"2024-09-15T21:00:50.567844+0000","last_fullsized":"2024-09-15T21:00:50.567844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:45.783884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017917219999999999,"stat_sum":{"num_bytes":6242304,"num_objects":6,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":18,"num_read_kb":11,"num_write":31,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.674822+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:00:50.674822+0000","last_peered":"2024-09-15T21:00:50.674822+0000","last_clean":"2024-09-15T21:00:50.674822+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:00:50.674822+0000","last_undegraded":"2024-09-15T21:00:50.674822+0000","last_fullsized":"2024-09-15T21:00:50.674822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"57'14","reported_seq":159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:00:49.675697+0000","last_change":"2024-09-15T21:00:47.650146+0000","last_active":"2024-09-15T21:00:49.675697+0000","last_peered":"2024-09-15T21:00:49.675697+0000","last_clean":"2024-09-15T21:00:49.675697+0000","last_became_active":"2024-09-15T21:00:20.315752+0000","last_became_peered":"2024-09-15T21:00:20.315752+0000","last_unstale":"2024-09-15T21:00:49.675697+0000","last_undegraded":"2024-09-15T21:00:49.675697+0000","last_fullsized":"2024-09-15T21:00:49.675697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:23.930266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071097300000000005,"stat_sum":{"num_bytes":2736128,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2029,"num_write":7,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":81,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:00:46.540947+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:00:46.540947+0000","last_peered":"2024-09-15T21:00:46.540947+0000","last_clean":"2024-09-15T21:00:46.540947+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:00:46.540947+0000","last_undegraded":"2024-09-15T21:00:46.540947+0000","last_fullsized":"2024-09-15T21:00:46.540947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:14.955970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"53'15","reported_seq":154,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:00:47.555587+0000","last_change":"2024-09-15T21:00:47.556009+0000","last_active":"2024-09-15T21:00:47.555587+0000","last_peered":"2024-09-15T21:00:47.555587+0000","last_clean":"2024-09-15T21:00:47.555587+0000","last_became_active":"2024-09-15T21:00:20.517607+0000","last_became_peered":"2024-09-15T21:00:20.517607+0000","last_unstale":"2024-09-15T21:00:47.555587+0000","last_undegraded":"2024-09-15T21:00:47.555587+0000","last_fullsized":"2024-09-15T21:00:47.555587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:34.301172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000790261,"stat_sum":{"num_bytes":3309568,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2808,"num_write":14,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"5"},{"start":"c","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:00:46.701108+0000","last_change":"2024-09-15T21:00:07.906899+0000","last_active":"2024-09-15T21:00:46.701108+0000","last_peered":"2024-09-15T21:00:46.701108+0000","last_clean":"2024-09-15T21:00:46.701108+0000","last_became_active":"2024-09-15T21:00:05.980693+0000","last_became_peered":"2024-09-15T21:00:05.980693+0000","last_unstale":"2024-09-15T21:00:46.701108+0000","last_undegraded":"2024-09-15T21:00:46.701108+0000","last_fullsized":"2024-09-15T21:00:46.701108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.412929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"52'24","reported_seq":169,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.575817+0000","last_change":"2024-09-15T21:00:47.560324+0000","last_active":"2024-09-15T21:00:50.575817+0000","last_peered":"2024-09-15T21:00:50.575817+0000","last_clean":"2024-09-15T21:00:50.575817+0000","last_became_active":"2024-09-15T21:00:20.410469+0000","last_became_peered":"2024-09-15T21:00:20.410469+0000","last_unstale":"2024-09-15T21:00:50.575817+0000","last_undegraded":"2024-09-15T21:00:50.575817+0000","last_fullsized":"2024-09-15T21:00:50.575817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:27.223422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028492050000000001,"stat_sum":{"num_bytes":1982464,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2317,"num_write":29,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"3.5","version":"53'71","reported_seq":216,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:00:46.602582+0000","last_change":"2024-09-15T21:00:43.765150+0000","last_active":"2024-09-15T21:00:46.602582+0000","last_peered":"2024-09-15T21:00:46.602582+0000","last_clean":"2024-09-15T21:00:41.175875+0000","last_became_active":"2024-09-15T21:00:43.559928+0000","last_became_peered":"2024-09-15T21:00:43.559928+0000","last_unstale":"2024-09-15T21:00:46.602582+0000","last_undegraded":"2024-09-15T21:00:43.162435+0000","last_fullsized":"2024-09-15T21:00:43.159796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40769283499999998,"stat_sum":{"num_bytes":9895936,"num_objects":10,"num_object_clones":2,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":49,"num_read_kb":6490,"num_write":80,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,2147483647,11,10],"avail_no_missing":["5(0)","1(5)","2(2)","3(1)","6(3)","7(4)","10(7)","11(6)"],"object_location_counts":[{"shards":"1(5),2(2),3(1),5(0),6(3),7(4),10(7),11(6)","objects":8},{"shards":"2(2),3(1),5(0),6(3),7(4),10(7),11(6)","objects":2}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:00:49.675851+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:00:49.675851+0000","last_peered":"2024-09-15T21:00:49.675851+0000","last_clean":"2024-09-15T21:00:49.675851+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:00:49.675851+0000","last_undegraded":"2024-09-15T21:00:49.675851+0000","last_fullsized":"2024-09-15T21:00:49.675851+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:07.395326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"52'66","reported_seq":224,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:00:47.570049+0000","last_change":"2024-09-15T21:00:45.262739+0000","last_active":"2024-09-15T21:00:47.570049+0000","last_peered":"2024-09-15T21:00:47.570049+0000","last_clean":"2024-09-15T21:00:40.357398+0000","last_became_active":"2024-09-15T21:00:43.581157+0000","last_became_peered":"2024-09-15T21:00:43.581157+0000","last_unstale":"2024-09-15T21:00:47.570049+0000","last_undegraded":"2024-09-15T21:00:43.163722+0000","last_fullsized":"2024-09-15T21:00:43.154527+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40577698000000001,"stat_sum":{"num_bytes":7094272,"num_objects":9,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":20,"num_read_kb":11,"num_write":37,"num_write_kb":8704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":5128192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[2147483647,11,9,5,2147483647,2,6,0],"avail_no_missing":["11(1)","0(7)","1(0)","2(5)","5(3)","6(6)","9(2)","10(4)"],"object_location_counts":[{"shards":"0(7),1(0),2(5),5(3),6(6),9(2),10(4),11(1)","objects":1},{"shards":"0(7),2(5),5(3),6(6),9(2),11(1)","objects":7}],"blocked_by":[],"up_primary":10,"acting_primary":11,"purged_snaps":[{"start":"3","length":"1"},{"start":"6","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.576940+0000","last_change":"2024-09-15T21:00:08.265808+0000","last_active":"2024-09-15T21:00:50.576940+0000","last_peered":"2024-09-15T21:00:50.576940+0000","last_clean":"2024-09-15T21:00:50.576940+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:00:50.576940+0000","last_undegraded":"2024-09-15T21:00:50.576940+0000","last_fullsized":"2024-09-15T21:00:50.576940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.117071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"34'5","reported_seq":150,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.567638+0000","last_change":"2024-09-15T21:00:47.578398+0000","last_active":"2024-09-15T21:00:50.567638+0000","last_peered":"2024-09-15T21:00:50.567638+0000","last_clean":"2024-09-15T21:00:50.567638+0000","last_became_active":"2024-09-15T21:00:20.313299+0000","last_became_peered":"2024-09-15T21:00:20.313299+0000","last_unstale":"2024-09-15T21:00:50.567638+0000","last_undegraded":"2024-09-15T21:00:50.567638+0000","last_fullsized":"2024-09-15T21:00:50.567638+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:39.493943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025555759999999999,"stat_sum":{"num_bytes":1490944,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.575331+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:00:50.575331+0000","last_peered":"2024-09-15T21:00:50.575331+0000","last_clean":"2024-09-15T21:00:50.575331+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:00:50.575331+0000","last_undegraded":"2024-09-15T21:00:50.575331+0000","last_fullsized":"2024-09-15T21:00:50.575331+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:16.994661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"58'22","reported_seq":196,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:00:49.579716+0000","last_change":"2024-09-15T21:00:49.575950+0000","last_active":"2024-09-15T21:00:49.579716+0000","last_peered":"2024-09-15T21:00:49.579716+0000","last_clean":"2024-09-15T21:00:49.579716+0000","last_became_active":"2024-09-15T21:00:48.749851+0000","last_became_peered":"2024-09-15T21:00:48.749851+0000","last_unstale":"2024-09-15T21:00:49.579716+0000","last_undegraded":"2024-09-15T21:00:49.579716+0000","last_fullsized":"2024-09-15T21:00:49.579716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:41.026459+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.82409034299999995,"stat_sum":{"num_bytes":3653632,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":960,"num_write":33,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2473984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"4"},{"start":"c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:00:50.670399+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:00:50.670399+0000","last_peered":"2024-09-15T21:00:50.670399+0000","last_clean":"2024-09-15T21:00:50.670399+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:00:50.670399+0000","last_undegraded":"2024-09-15T21:00:50.670399+0000","last_fullsized":"2024-09-15T21:00:50.670399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:06.832542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":52183040,"num_objects":60,"num_object_clones":14,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":3,"num_read":216,"num_read_kb":25424,"num_write":372,"num_write_kb":58832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":15499264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":109305856,"data_stored":127873024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":398,"ondisk_log_size":398,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378638,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":5580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":5713920,"data_stored":6253692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411342,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41908,"kb_used_data":10712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329932,"statfs":{"total":96636764160,"available":96593850368,"internally_reserved":0,"allocated":10969088,"data_stored":12082844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411341,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41496,"kb_used_data":10308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330344,"statfs":{"total":96636764160,"available":96594272256,"internally_reserved":0,"allocated":10555392,"data_stored":11181180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411341,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":9548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":9777152,"data_stored":11039112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411341,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37644,"kb_used_data":10620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334196,"statfs":{"total":96636764160,"available":96598216704,"internally_reserved":0,"allocated":10874880,"data_stored":13067176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444046,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38260,"kb_used_data":11232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333580,"statfs":{"total":96636764160,"available":96597585920,"internally_reserved":0,"allocated":11501568,"data_stored":12894600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444047,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40452,"kb_used_data":13424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331388,"statfs":{"total":96636764160,"available":96595341312,"internally_reserved":0,"allocated":13746176,"data_stored":15905160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444045,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40500,"kb_used_data":9304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331340,"statfs":{"total":96636764160,"available":96595292160,"internally_reserved":0,"allocated":9527296,"data_stored":10398389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444045,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34324,"kb_used_data":7296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337516,"statfs":{"total":96636764160,"available":96601616384,"internally_reserved":0,"allocated":7471104,"data_stored":8696200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444047,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35796,"kb_used_data":8768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336044,"statfs":{"total":96636764160,"available":96600109056,"internally_reserved":0,"allocated":8978432,"data_stored":10297736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444045,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":10364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":10612736,"data_stored":12161510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444046,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35032,"kb_used_data":8008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336808,"statfs":{"total":96636764160,"available":96600891392,"internally_reserved":0,"allocated":8200192,"data_stored":9732563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7925760,"data_stored":9814016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8269824,"data_stored":9908224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":5898240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8310784,"data_stored":9908224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7168000,"data_stored":8187904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8916992,"data_stored":10051584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13078528,"data_stored":15515648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9613312,"data_stored":12087296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":9109504,"data_stored":10567680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":11509760,"data_stored":13197312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10489856,"data_stored":11538432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9818112,"data_stored":11198464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:00:52.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 3.5 3.4 2024-09-15T21:00:52.113 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:52.107+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:52.123 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:52.124 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:52.147 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:52.147 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:52.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 60/28 ECSubWriteReply(tid=60, last_complete=60'39, committed=1, applied=1))", 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:52.110530+0000", 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.036798392999999999, 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036835031999999997, 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110530+0000", 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110530+0000", 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110531+0000", 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8369999999999999e-06 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110544+0000", 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2733e-05 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110573+0000", 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312999999999999e-05 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110592+0000", 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8425000000000001e-05 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 60/28 ECSubWriteReply(tid=60, last_complete=60'39, committed=1, applied=1))", 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:52.110905+0000", 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.036423206999999999, 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036532724000000003, 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110905+0000", 2024-09-15T21:00:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110905+0000", 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110907+0000", 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.756e-06 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110917+0000", 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0413e-05 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110945+0000", 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8161999999999999e-05 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 60/28 ECSubWriteReply(tid=60, last_complete=60'39, committed=1, applied=1))", 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:52.110965+0000", 2024-09-15T21:00:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.036362618999999999, 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.036527589999999999, 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110965+0000", 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110965+0000", 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110967+0000", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0999999999999998e-06 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110979+0000", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1678999999999999e-05 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.111012+0000", 2024-09-15T21:00:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2555999999999998e-05 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.111035+0000", 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3586000000000001e-05 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 60/28 ECSubWriteReply(tid=60, last_complete=60'39, committed=1, applied=1))", 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:52.124632+0000", 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.022695856, 2024-09-15T21:00:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.022921271999999999, 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "no flag points reached", 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.124632+0000", 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.124632+0000", 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.124634+0000", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.674e-06 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.124642+0000", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6489999999999997e-06 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.124672+0000", 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9056000000000001e-05 2024-09-15T21:00:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 4 2024-09-15T21:00:52.159 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:52.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:52.213 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:52.208+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:52.232 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:52.253 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:52.253 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:52.275 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:52.275 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:52.286 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:00:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:00:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:00:52.287 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:00:52.295 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:52.295 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:52.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:52.318 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:52.318 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:52.335 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:52.336 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:52.346 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:52.344+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:52.355 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:52.356 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:52.375 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:52.375 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:52.396 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:52.415 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:52.411+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:52.440 INFO:teuthology.orchestra.run.smithi027.stderr:instructing pg(s) [3.5s0] on osd.5 to force-recovery; instructing pg(s) [3.4s0] on osd.10 to force-recovery; 2024-09-15T21:00:52.440 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:00:52.474 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:52.474 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:52.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:52.534 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:52.532+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:52.597 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:00:52.602 INFO:tasks.rados.rados.0.smithi027.stdout:181: done (7 left) 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:182: read oid 5 snap -1 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:182: expect deleted 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:183: delete oid 19 current snap is 14 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:177: finishing copy_from to smithi02732947-47 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:177: got expected ENOENT (src dne) 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:177: finishing copy_from racing read to smithi02732947-47 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:174: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-15T21:00:52.603 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 6169 2024-09-15T21:00:52.604 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:52.605 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:175: finishing copy_from racing read to smithi02732947-24 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:180: finishing rollback tid 0 to smithi02732947-22 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:180: finishing rollback tid 1 to smithi02732947-22 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:180: finishing rollback tid 2 to smithi02732947-22 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 39 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:179: finishing write tid 1 to smithi02732947-13 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:179: finishing write tid 2 to smithi02732947-13 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:175: finishing copy_from to smithi02732947-24 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 24 v 27 (ObjNum 63 snap 3 seq_num 63) dirty exists 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:176: finishing write tid 1 to smithi02732947-41 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:176: finishing write tid 2 to smithi02732947-41 2024-09-15T21:00:52.607 INFO:tasks.rados.rados.0.smithi027.stdout:176: finishing write tid 3 to smithi02732947-41 2024-09-15T21:00:52.608 INFO:tasks.rados.rados.0.smithi027.stdout:179: finishing write tid 3 to smithi02732947-13 2024-09-15T21:00:52.608 INFO:tasks.rados.rados.0.smithi027.stdout:179: finishing write tid 4 to smithi02732947-13 2024-09-15T21:00:52.612 INFO:tasks.rados.rados.0.smithi027.stdout:176: finishing write tid 4 to smithi02732947-41 2024-09-15T21:00:52.615 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:52.615 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:52.616 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:52.612+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:52.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:52.611+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:52.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:52.611+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:52.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:52.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:52.696+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:52.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:52.696+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:00:52.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:52.696+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,4,5 2024-09-15T21:00:52.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:52.700+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6,7 to read 0,1,4,5 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: finishing write tid 5 to smithi02732947-13 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: oid 13 updating version 0 to 83 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: oid 13 version 83 is already newer than 81 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: oid 13 version 83 is already newer than 82 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: oid 13 updating version 83 to 84 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 84 (ObjNum 88 snap 14 seq_num 88) dirty exists 2024-09-15T21:00:52.703 INFO:tasks.rados.rados.0.smithi027.stdout:179: left oid 13 (ObjNum 88 snap 14 seq_num 88) 2024-09-15T21:00:52.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:52.711+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4 to read 0,1,4,5 2024-09-15T21:00:52.718 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:52.712+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:52.718 INFO:tasks.rados.rados.0.smithi027.stdout:176: finishing write tid 5 to smithi02732947-41 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:176: oid 41 updating version 0 to 74 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:176: oid 41 updating version 74 to 76 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:176: oid 41 version 76 is already newer than 73 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:176: oid 41 version 76 is already newer than 75 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 76 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-15T21:00:52.719 INFO:tasks.rados.rados.0.smithi027.stdout:176: left oid 41 (ObjNum 87 snap 14 seq_num 87) 2024-09-15T21:00:52.800 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:52.800 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:52.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:52.818 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:52.816+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:52.919 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:52.916+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:52.985 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:52.985 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:53.020 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:53.016+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:53.120 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:53.116+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:53.166 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.166 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:174: done (8 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:175: done (7 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:176: done (6 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:177: done (5 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:179: done (4 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:180: done (3 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:182: done (2 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:183: done (1 left) 2024-09-15T21:00:53.207 INFO:tasks.rados.rados.0.smithi027.stdout:184: delete oid 38 current snap is 14 2024-09-15T21:00:53.221 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:53.215+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:53.321 INFO:tasks.rados.rados.0.smithi027.stdout:178: finishing rollback tid 0 to smithi02732947-25 2024-09-15T21:00:53.322 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:53.316+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:53.322 INFO:tasks.rados.rados.0.smithi027.stdout:178: finishing rollback tid 1 to smithi02732947-25 2024-09-15T21:00:53.323 INFO:tasks.rados.rados.0.smithi027.stdout:178: finishing rollback tid 2 to smithi02732947-25 2024-09-15T21:00:53.323 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 82 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T21:00:53.325 INFO:tasks.rados.rados.0.smithi027.stdout:178: done (1 left) 2024-09-15T21:00:53.325 INFO:tasks.rados.rados.0.smithi027.stdout:184: done (0 left) 2024-09-15T21:00:53.326 INFO:tasks.rados.rados.0.smithi027.stdout:185: setattr oid 29 current snap is 14 2024-09-15T21:00:53.329 INFO:tasks.rados.rados.0.smithi027.stdout:186: snap_remove snap 9 2024-09-15T21:00:53.345 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.346 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:53.422 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:53.420+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:53.523 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:53.519+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:53.525 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.526 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:00:53.619 INFO:tasks.rados.rados.0.smithi027.stdout:186: done (1 left) 2024-09-15T21:00:53.619 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 38 current snap is 14 2024-09-15T21:00:53.619 INFO:tasks.rados.rados.0.smithi027.stdout:187: seq_num 91 ranges {0=770048,770048=475136,1245184=294912} 2024-09-15T21:00:53.624 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:53.620+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:53.632 INFO:tasks.rados.rados.0.smithi027.stdout:187: writing smithi02732947-38 from 0 to 770048 tid 1 2024-09-15T21:00:53.642 INFO:tasks.rados.rados.0.smithi027.stdout:187: writing smithi02732947-38 from 770048 to 1245184 tid 2 2024-09-15T21:00:53.647 INFO:tasks.rados.rados.0.smithi027.stdout:187: writing smithi02732947-38 from 1245184 to 1540096 tid 3 2024-09-15T21:00:53.648 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 49 current snap is 14 2024-09-15T21:00:53.648 INFO:tasks.rados.rados.0.smithi027.stdout:188: seq_num 92 ranges {2015232=786432,2801664=491520,3293184=278528} 2024-09-15T21:00:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.688 INFO:tasks.rados.rados.0.smithi027.stdout:188: writing smithi02732947-49 from 2015232 to 2801664 tid 1 2024-09-15T21:00:53.693 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:00:53.694 INFO:tasks.rados.rados.0.smithi027.stdout:188: writing smithi02732947-49 from 2801664 to 3293184 tid 2 2024-09-15T21:00:53.696 INFO:tasks.rados.rados.0.smithi027.stdout:188: writing smithi02732947-49 from 3293184 to 3571712 tid 3 2024-09-15T21:00:53.696 INFO:tasks.rados.rados.0.smithi027.stdout:189: copy_from oid 26 from oid 14 current snap is 14 2024-09-15T21:00:53.696 INFO:tasks.rados.rados.0.smithi027.stdout:190: snap_create 2024-09-15T21:00:53.734 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:53.732+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:53.826 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:53.820+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:53.837 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 29 v 36 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T21:00:53.851 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.852 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:53.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:00:53.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:53.923+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:53.949 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:53.944+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:53.994 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:53.994 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:54.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:00:54.027 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:54.023+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:54.049 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.044+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:00:54.049 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.044+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,4,5 2024-09-15T21:00:54.128 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:00:54.124+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:00:54.147 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:54.147 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:54.152 INFO:tasks.rados.rados.0.smithi027.stdout:188: finishing write tid 1 to smithi02732947-49 2024-09-15T21:00:54.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:00:54.250 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:54.248+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:54.253 INFO:tasks.rados.rados.0.smithi027.stdout:189: finishing copy_from racing read to smithi02732947-26 2024-09-15T21:00:54.254 INFO:tasks.rados.rados.0.smithi027.stdout:187: finishing write tid 1 to smithi02732947-38 2024-09-15T21:00:54.254 INFO:tasks.rados.rados.0.smithi027.stdout:187: finishing write tid 2 to smithi02732947-38 2024-09-15T21:00:54.254 INFO:tasks.rados.rados.0.smithi027.stdout:187: finishing write tid 3 to smithi02732947-38 2024-09-15T21:00:54.255 INFO:tasks.rados.rados.0.smithi027.stdout:187: finishing write tid 4 to smithi02732947-38 2024-09-15T21:00:54.330 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:54.328+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:54.334 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:54.335 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:54.346 INFO:tasks.rados.rados.0.smithi027.stdout:188: finishing write tid 2 to smithi02732947-49 2024-09-15T21:00:54.347 INFO:tasks.rados.rados.0.smithi027.stdout:188: finishing write tid 3 to smithi02732947-49 2024-09-15T21:00:54.350 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:00:54.350 INFO:tasks.rados.rados.0.smithi027.stdout:188: finishing write tid 4 to smithi02732947-49 2024-09-15T21:00:54.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:54.434 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:54.432+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:54.450 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.448+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:00:54.525 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:54.525 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:54.531 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:54.528+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:54.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:00:54.558 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:54.556+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:54.558 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:54.556+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:00:54.559 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:54.556+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: finishing write tid 5 to smithi02732947-38 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: oid 38 updating version 0 to 97 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: oid 38 version 97 is already newer than 96 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: oid 38 version 97 is already newer than 95 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: oid 38 updating version 97 to 98 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 98 (ObjNum 91 snap 14 seq_num 91) dirty exists 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:187: left oid 38 (ObjNum 91 snap 14 seq_num 91) 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:189: finishing copy_from to smithi02732947-26 2024-09-15T21:00:54.560 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 26 v 10 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T21:00:54.579 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:54.598 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:54.598 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:54.614 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:54.615 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:54.618 INFO:tasks.rados.rados.0.smithi027.stdout:185: done (4 left) 2024-09-15T21:00:54.618 INFO:tasks.rados.rados.0.smithi027.stdout:187: done (3 left) 2024-09-15T21:00:54.618 INFO:tasks.rados.rados.0.smithi027.stdout:189: done (2 left) 2024-09-15T21:00:54.618 INFO:tasks.rados.rados.0.smithi027.stdout:190: done (1 left) 2024-09-15T21:00:54.618 INFO:tasks.rados.rados.0.smithi027.stdout:191: snap_create 2024-09-15T21:00:54.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:00:54.632 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:54.632 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:54.633 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:54.627+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:54.652 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.648+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:00:54.652 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.648+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:00:54.652 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:54.648+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,7 to read 0,1,4,5 2024-09-15T21:00:54.653 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:54.653 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:54.675 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:54.680 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:54.680 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:54.680 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:54.680 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:54.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:00:54.734 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:54.728+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:54.752 INFO:tasks.rados.rados.0.smithi027.stdout:188: finishing write tid 5 to smithi02732947-49 2024-09-15T21:00:54.752 INFO:tasks.rados.rados.0.smithi027.stdout:188: oid 49 updating version 0 to 59 2024-09-15T21:00:54.753 INFO:tasks.rados.rados.0.smithi027.stdout:188: oid 49 updating version 59 to 61 2024-09-15T21:00:54.753 INFO:tasks.rados.rados.0.smithi027.stdout:188: oid 49 version 61 is already newer than 58 2024-09-15T21:00:54.753 INFO:tasks.rados.rados.0.smithi027.stdout:188: oid 49 version 61 is already newer than 60 2024-09-15T21:00:54.753 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 49 v 61 (ObjNum 92 snap 14 seq_num 92) dirty exists 2024-09-15T21:00:54.753 INFO:tasks.rados.rados.0.smithi027.stdout:188: left oid 49 (ObjNum 92 snap 14 seq_num 92) 2024-09-15T21:00:54.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:54.783 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:54.804 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:54.825 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:54.826 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:54.835 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:54.832+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:54.847 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:54.848 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:54.861 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:54.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:54.861 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:54.861 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:54.861 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:54.870 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:54.872 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:54.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:00:54.936 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:54.931+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:54.979 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:54.979 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:55.037 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:55.032+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.290147888, 2024-09-15T21:00:55.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.29010083, 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:00:55.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:00:55.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.290073029999999, 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.061 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.290047636000001, 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:55.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:00:55.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:55.065 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.984668414000001, 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:00:55.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:00:55.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:00:55.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.984107454, 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.071 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:00:55.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:00:55.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.883949071, 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:00:55.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:00:55.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:00:55.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.883852601000001, 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.082 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:00:55.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:00:55.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.882895171000001, 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.088 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:00:55.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:00:55.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.882825664999999, 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:00:55.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:00:55.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.783057960000001, 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.102 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.679385787000001, 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:55.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:55.106 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.666126016, 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.664185588999999, 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:00:55.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.474531962, 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:55.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:00:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.065521532999998, 2024-09-15T21:00:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:00:55.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.058000125, 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.124 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.865120515000001, 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.128 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.864117437000001, 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:00:55.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.199564632000001, 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:00:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.136 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.136 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:55.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:00:55.137 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:55.136+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:55.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:55.235+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:55.311 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:55.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:55.312 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:55.312 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:55.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:00:55.340 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:55.336+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:55.441 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:55.436+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:55.452 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:55.452 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:55.452 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:55.452 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:55.453 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:55.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:00:55.555 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:55.552+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:55.625 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:00:55.632 INFO:tasks.rados.rados.0.smithi027.stdout:188: done (1 left) 2024-09-15T21:00:55.632 INFO:tasks.rados.rados.0.smithi027.stdout:191: done (0 left) 2024-09-15T21:00:55.632 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 17 current snap is 16 2024-09-15T21:00:55.632 INFO:tasks.rados.rados.0.smithi027.stdout:192: seq_num 93 ranges {835584=802816,1638400=638976,2277376=131072} 2024-09-15T21:00:55.643 INFO:tasks.rados.rados.0.smithi027.stdout:192: writing smithi02732947-17 from 835584 to 1638400 tid 1 2024-09-15T21:00:55.643 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:55.640+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:55.647 INFO:tasks.rados.rados.0.smithi027.stdout:192: writing smithi02732947-17 from 1638400 to 2277376 tid 2 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:192: writing smithi02732947-17 from 2277376 to 2408448 tid 3 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:193: rollback oid 8 current snap is 16 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 8 to 14 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:194: rmattr oid 26 current snap is 16 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:194: done (2 left) 2024-09-15T21:00:55.649 INFO:tasks.rados.rados.0.smithi027.stdout:195: delete oid 2 current snap is 16 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.654 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 33/28 ECSubWrite(tid=17, reqid=client.4641.0:106, at_version=33'11, trim_to=0'0, roll_forward_to=32'10))", 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.370145291, 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103549026, 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280136+0000", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280137+0000", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280251+0000", 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000113705 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280273+0000", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2198e-05 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.280285+0000", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2014999999999999e-05 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381500+0000", 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10121572400000001 2024-09-15T21:00:55.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.381526+0000", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5077000000000001e-05 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383596+0000", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020703029999999999 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.383685+0000", 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9223999999999998e-05 2024-09-15T21:00:55.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=29, reqid=client.4641.0:124, at_version=33'11, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.072908454, 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103593675, 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577373+0000", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577375+0000", 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9870000000000002e-06 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577857+0000", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048242599999999999 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577888+0000", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1007000000000003e-05 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577904+0000", 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5588000000000001e-05 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578065+0000", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000161593 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578111+0000", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5977999999999999e-05 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680842+0000", 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102730619 2024-09-15T21:00:55.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680967+0000", 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012447700000000001 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=30, reqid=client.4641.0:125, at_version=33'12, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.072337083000001, 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10310723500000001, 2024-09-15T21:00:55.663 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577944+0000", 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577945+0000", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.096e-06 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577980+0000", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4836000000000003e-05 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.577999+0000", 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8975e-05 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578040+0000", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1328999999999997e-05 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579237+0000", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011965879999999999 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.579269+0000", 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.222e-05 2024-09-15T21:00:55.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.680983+0000", 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10171382499999999 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681051+0000", 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8366000000000002e-05 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 33/28 ECSubWrite(tid=31, reqid=client.4641.0:130, at_version=33'13, trim_to=0'0, roll_forward_to=32'8))", 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.072209869999998, 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10373313300000001, 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578071+0000", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578073+0000", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.319e-06 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578381+0000", 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000308282 2024-09-15T21:00:55.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578400+0000", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8788e-05 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.578412+0000", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2509e-05 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679722+0000", 2024-09-15T21:00:55.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10130917 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.679753+0000", 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1386000000000001e-05 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681744+0000", 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019913180000000002 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.681805+0000", 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0361e-05 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.054730048, 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.672 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:00:55.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:00:55.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.566777659, 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:55.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:00:55.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:00:55.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:00:55.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.863764028999999, 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:00:55.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=79, reqid=osd.5.0:79, at_version=39'14, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.338191749, 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103073317, 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312090+0000", 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7000000000000005e-07 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312125+0000", 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4603000000000001e-05 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312135+0000", 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.04e-05 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312143+0000", 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.3950000000000003e-06 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312254+0000", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000111611 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312269+0000", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4788e-05 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415084+0000", 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102815383 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415163+0000", 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8566999999999995e-05 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=80, reqid=client.4641.0:250, at_version=39'15, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.338120573000001, 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10304223699999999, 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.692 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312161+0000", 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312378+0000", 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00021675199999999999 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312400+0000", 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2379e-05 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312409+0000", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0939999999999997e-06 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.412940+0000", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10053021500000001 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.412958+0000", 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8535000000000001e-05 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415170+0000", 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022119370000000002 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415203+0000", 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2947000000000002e-05 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.696 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=81, reqid=client.4641.0:251, at_version=39'16, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.337854566000001, 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102905732, 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312427+0000", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1300000000000001e-07 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312521+0000", 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3363999999999996e-05 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312533+0000", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2804e-05 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312542+0000", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0149999999999992e-06 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.413828+0000", 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101285968 2024-09-15T21:00:55.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.413844+0000", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5849e-05 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415302+0000", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0014583420000000001 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415332+0000", 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9977000000000001e-05 2024-09-15T21:00:55.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 39/28 ECSubWrite(tid=82, reqid=client.4641.0:252, at_version=39'17, trim_to=0'0, roll_forward_to=38'12))", 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.337725004999999, 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.103356555, 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312556+0000", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312557+0000", 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4900000000000001e-07 2024-09-15T21:00:55.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312587+0000", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0343000000000001e-05 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312596+0000", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.312602+0000", 2024-09-15T21:00:55.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3459999999999999e-06 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.414704+0000", 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102101729 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.414731+0000", 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7211000000000001e-05 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415849+0000", 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011185909999999999 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.415913+0000", 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3417999999999995e-05 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:00:55.705 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.277640730999998, 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104322594, 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.706 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372642+0000", 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7069999999999999e-06 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372856+0000", 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000213255 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372963+0000", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107661 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473110+0000", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100146978 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473309+0000", 2024-09-15T21:00:55.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019897599999999999 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473331+0000", 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1427999999999999e-05 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476903+0000", 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035721609999999999 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476963+0000", 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0427999999999998e-05 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 48/28 ECSubRead(tid=132, to_read={3:a5da2594:::smithi02732947-25:head=0,315392,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.187246545000001, 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20088298299999999, 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.710 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:00:55.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463043+0000", 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463058+0000", 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4732999999999999e-05 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.563142+0000", 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008391699999999 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663346+0000", 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100204423 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663918+0000", 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057127899999999995 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.713 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.762758+0000", 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8875231169999998, 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10331775999999999, 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.762758+0000", 2024-09-15T21:00:55.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.762758+0000", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.762760+0000", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.863e-06 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.763415+0000", 2024-09-15T21:00:55.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000655194 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.763447+0000", 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1482000000000001e-05 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.863602+0000", 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015560699999999 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.863738+0000", 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013580600000000001 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.863754+0000", 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5478999999999999e-05 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.865993+0000", 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022397530000000001 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.866076+0000", 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2576000000000004e-05 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.779222989, 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30162102800000001, 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871059+0000", 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871100+0000", 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0321999999999998e-05 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871128+0000", 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8208000000000002e-05 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871152+0000", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4246000000000001e-05 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971464+0000", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312292 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971482+0000", 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7176e-05 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172613+0000", 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20113102099999999 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172679+0000", 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6762999999999998e-05 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.722 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.7789181300000001, 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20089554500000001, 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871364+0000", 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871378+0000", 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3706000000000001e-05 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871408+0000", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0574999999999998e-05 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971547+0000", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013905400000001 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072019+0000", 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100471385 2024-09-15T21:00:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072259+0000", 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023990200000000001 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7512846180000001, 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20140428699999999, 2024-09-15T21:00:55.726 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898998+0000", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4980000000000001e-06 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899107+0000", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010874699999999999 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899130+0000", 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.268e-05 2024-09-15T21:00:55.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999226+0000", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096764 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999366+0000", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139231 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999377+0000", 2024-09-15T21:00:55.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1097000000000001e-05 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100334+0000", 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100957065 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100401+0000", 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7205000000000005e-05 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=180, reqid=client.4641.0:451, at_version=60'74, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.7495060589999998, 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.200850169, 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900776+0000", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900867+0000", 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1712000000000006e-05 2024-09-15T21:00:55.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900880+0000", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.218e-05 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001054+0000", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10017451600000001 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001292+0000", 2024-09-15T21:00:55.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023806500000000001 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001309+0000", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101581+0000", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027267099999999 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101625+0000", 2024-09-15T21:00:55.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3965999999999999e-05 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=182, reqid=client.4641.0:453, at_version=60'76, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.6490503470000002, 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10391302099999999, 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001232+0000", 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7599999999999997e-07 2024-09-15T21:00:55.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001254+0000", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2178e-05 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001269+0000", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5576000000000001e-05 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001282+0000", 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2642e-05 2024-09-15T21:00:55.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101230+0000", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099947724000000002 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101250+0000", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105105+0000", 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038547550000000001 2024-09-15T21:00:55.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105144+0000", 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9075000000000001e-05 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 60/28 ECSubWrite(tid=183, reqid=client.4641.0:448, at_version=60'44, trim_to=0'0, roll_forward_to=56'42))", 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.6489800089999997, 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104610525, 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:55.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001302+0000", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9500000000000002e-07 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001809+0000", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050734700000000001 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001828+0000", 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8417999999999999e-05 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101908+0000", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008057200000001 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101977+0000", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8571999999999993e-05 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101983+0000", 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1990000000000003e-06 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105842+0000", 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038594150000000002 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105912+0000", 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9406999999999995e-05 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:55.743 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:55.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:00:55.746 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:55.740+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:55.845 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:55.840+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:55.862 INFO:tasks.rados.rados.0.smithi027.stdout:193: finishing rollback tid 0 to smithi02732947-8 2024-09-15T21:00:55.918 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:55.918 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:55.918 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:00:55.918 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:55.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:00:55.946 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:55.944+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:55.954 INFO:tasks.rados.rados.0.smithi027.stdout:192: finishing write tid 1 to smithi02732947-17 2024-09-15T21:00:55.962 INFO:tasks.rados.rados.0.smithi027.stdout:193: finishing rollback tid 1 to smithi02732947-8 2024-09-15T21:00:55.963 INFO:tasks.rados.rados.0.smithi027.stdout:193: finishing rollback tid 2 to smithi02732947-8 2024-09-15T21:00:55.963 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 39 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T21:00:56.048 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:56.044+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:56.052 INFO:tasks.rados.rados.0.smithi027.stdout:193: done (2 left) 2024-09-15T21:00:56.053 INFO:tasks.rados.rados.0.smithi027.stdout:195: done (1 left) 2024-09-15T21:00:56.053 INFO:tasks.rados.rados.0.smithi027.stdout:196: rmattr oid 10 current snap is 16 2024-09-15T21:00:56.053 INFO:tasks.rados.rados.0.smithi027.stdout:196: done (1 left) 2024-09-15T21:00:56.053 INFO:tasks.rados.rados.0.smithi027.stdout:197: snap_create 2024-09-15T21:00:56.085 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:56.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:00:56.085 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:00:56.085 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:00:56.085 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:56.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:00:56.148 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:56.144+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:56.152 INFO:tasks.rados.rados.0.smithi027.stdout:192: finishing write tid 2 to smithi02732947-17 2024-09-15T21:00:56.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:56.243+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:56.253 INFO:tasks.rados.rados.0.smithi027.stdout:192: finishing write tid 3 to smithi02732947-17 2024-09-15T21:00:56.253 INFO:tasks.rados.rados.0.smithi027.stdout:192: finishing write tid 4 to smithi02732947-17 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.272 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.292332539999997, 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.273 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:00:56.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:00:56.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.290321294999998, 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:00:56.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:00:56.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.289569641, 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.283 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.285 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.286 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:00:56.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.288 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 32.289503457000002, 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.289 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.290 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.291 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.292 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.293 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:129 3.7s0 3:e8b6cabe:::smithi02732947-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.990398171999999, 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61157217500000005, 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 129 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.294 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278368+0000", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278370+0000", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6129999999999998e-06 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278383+0000", 2024-09-15T21:00:56.295 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3026e-05 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278414+0000", 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0857e-05 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278434+0000", 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9924000000000001e-05 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.296 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.366928+0000", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088493789000000003 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367150+0000", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000222081 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690379+0000", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32322944100000001 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.297 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.690606+0000", 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022645599999999999 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.889940+0000", 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19933398799999999 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:56.298 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.784242632000002, 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.299 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.301 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.302 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:00:56.303 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.782805681999999, 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:00:56.306 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.307 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.308 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:00:56.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.310 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.782727537, 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.311 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.312 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.313 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.314 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.680212403999999, 2024-09-15T21:00:56.315 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.316 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:00:56.317 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.318 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:00:56.319 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:00:56.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.321 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.600165677, 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:00:56.322 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:56.323 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.326 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:00:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:00:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.327 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:00:56.328 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:00:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:00:56.329 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.330 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.585777586999999, 2024-09-15T21:00:56.331 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.332 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:00:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.333 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.334 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:00:56.335 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:00:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:00:56.336 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:00:56.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:00:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:00:56.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.339 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.340 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.341 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:00:56.342 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:00:56.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.344 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.579367632, 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.346 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.347 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.348 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.348 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:00:56.348 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.348 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.349 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.350 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:00:56.351 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:00:56.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.351 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.351 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.351 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:00:56.352 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:00:56.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.354 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.354 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.354 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:00:56.354 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:00:56.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.355 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.355 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:00:56.355 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:00:56.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.358 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.358 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.358 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:56.358 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.480719869000001, 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60734763000000003, 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.360 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.361 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.361 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788046+0000", 2024-09-15T21:00:56.361 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788048+0000", 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.063e-06 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.362 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788069+0000", 2024-09-15T21:00:56.363 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1321000000000001e-05 2024-09-15T21:00:56.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.363 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.363 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.363 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.788097+0000", 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8136999999999998e-05 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888193+0000", 2024-09-15T21:00:56.364 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100095434 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394534+0000", 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50634117999999995 2024-09-15T21:00:56.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395394+0000", 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00085949500000000005 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.366 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.367 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.367 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:00:56.367 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.478814659000001, 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.370 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:00:56.371 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.372 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:00:56.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.374 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:00:56.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.376 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.396857281999999, 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62454644199999998, 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:56.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871910+0000", 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.55e-06 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871958+0000", 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7698000000000003e-05 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871989+0000", 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1061e-05 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.872007+0000", 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7934000000000001e-05 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496399+0000", 2024-09-15T21:00:56.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62439274499999997 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496455+0000", 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5454000000000001e-05 2024-09-15T21:00:56.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:00:56.385 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.379989842000001, 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60786671999999997, 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888776+0000", 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888777+0000", 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4e-07 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888795+0000", 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8403999999999999e-05 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888808+0000", 2024-09-15T21:00:56.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2359e-05 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.888816+0000", 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3359999999999995e-06 2024-09-15T21:00:56.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496583+0000", 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60776654399999996 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496643+0000", 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0136999999999997e-05 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:160 3.2s0 3:48a96439:::smithi02732947-3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e33)", 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.975749540999999, 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61085863699999998, 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 160 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293016+0000", 2024-09-15T21:00:56.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293018+0000", 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.074e-06 2024-09-15T21:00:56.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293027+0000", 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3820000000000004e-06 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293049+0000", 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2697999999999999e-05 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.293072+0000", 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671e-05 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699705+0000", 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406633417 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.699975+0000", 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026989499999999998 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.701228+0000", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012527560000000001 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.702476+0000", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001248033 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903693+0000", 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20121672600000001 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.903875+0000", 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018198500000000001 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.068191070000001, 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:00:56.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:00:56.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:00:56.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.965237119999999, 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66775810800000002, 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303530+0000", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8299999999999995e-07 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303918+0000", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038879600000000001 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303931+0000", 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-05 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303938+0000", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4359999999999999e-06 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971275+0000", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66733690499999998 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971287+0000", 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2007999999999999e-05 2024-09-15T21:00:56.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.958010535, 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66056677200000002, 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:00:56.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310756+0000", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:00:56.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311135+0000", 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037949099999999998 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311146+0000", 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0736e-05 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311151+0000", 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3299999999999998e-06 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971311+0000", 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66015986699999996 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971322+0000", 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:00:56.412 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:00:56.414 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:56.351+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:56.414 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:56.351+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:56.414 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:56.351+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:56.415 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:00:56.415 INFO:tasks.rados.rados.0.smithi027.stdout:192: finishing write tid 5 to smithi02732947-17 2024-09-15T21:00:56.415 INFO:tasks.rados.rados.0.smithi027.stdout:192: oid 17 updating version 0 to 92 2024-09-15T21:00:56.415 INFO:tasks.rados.rados.0.smithi027.stdout:192: oid 17 version 92 is already newer than 90 2024-09-15T21:00:56.415 INFO:tasks.rados.rados.0.smithi027.stdout:192: oid 17 version 92 is already newer than 91 2024-09-15T21:00:56.416 INFO:tasks.rados.rados.0.smithi027.stdout:192: oid 17 updating version 92 to 93 2024-09-15T21:00:56.416 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 17 v 93 (ObjNum 93 snap 16 seq_num 93) dirty exists 2024-09-15T21:00:56.416 INFO:tasks.rados.rados.0.smithi027.stdout:192: left oid 17 (ObjNum 93 snap 16 seq_num 93) 2024-09-15T21:00:56.416 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:56.412+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:00:56.515 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:56.511+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:56.551 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:56.552 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:56.552 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:56.552 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:56.566 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:00:56.640 INFO:tasks.rados.rados.0.smithi027.stdout:192: done (1 left) 2024-09-15T21:00:56.640 INFO:tasks.rados.rados.0.smithi027.stdout:197: done (0 left) 2024-09-15T21:00:56.641 INFO:tasks.rados.rados.0.smithi027.stdout:198: copy_from oid 48 from oid 47 current snap is 17 2024-09-15T21:00:56.641 INFO:tasks.rados.rados.0.smithi027.stdout:199: read oid 34 snap -1 2024-09-15T21:00:56.641 INFO:tasks.rados.rados.0.smithi027.stdout:199: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T21:00:56.641 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 22 current snap is 17 2024-09-15T21:00:56.641 INFO:tasks.rados.rados.0.smithi027.stdout:200: seq_num 95 ranges {2129920=442368,2572288=540672,3112960=622592} 2024-09-15T21:00:56.650 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.647+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:56.673 INFO:tasks.rados.rados.0.smithi027.stdout:200: writing smithi02732947-22 from 2129920 to 2572288 tid 1 2024-09-15T21:00:56.681 INFO:tasks.rados.rados.0.smithi027.stdout:200: writing smithi02732947-22 from 2572288 to 3112960 tid 2 2024-09-15T21:00:56.688 INFO:tasks.rados.rados.0.smithi027.stdout:200: writing smithi02732947-22 from 3112960 to 3735552 tid 3 2024-09-15T21:00:56.689 INFO:tasks.rados.rados.0.smithi027.stdout:201: snap_remove snap 14 2024-09-15T21:00:56.716 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:56.712+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:56.733 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:56.733 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:56.733 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:00:56.733 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:00:56.733 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:56.747 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:00:56.835 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.831+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:56.848 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.843+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.141497868999998, 2024-09-15T21:00:56.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:56.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:00:56.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.140396502999998, 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.921 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:00:56.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.140355999000001, 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:00:56.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.931 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.11980449, 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.932 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:00:56.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.051516892000002, 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:00:56.937 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:00:56.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.018497633999999, 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.941 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:00:56.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:00:56.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:56.946 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.018457243999997, 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:56.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.951 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.950245078999998, 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.948342273999998, 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.958 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.946283944999998, 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202146044, 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.959 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965234+0000", 2024-09-15T21:00:56.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7099999999999999e-06 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965259+0000", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5732999999999999e-05 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965292+0000", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2223e-05 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965310+0000", 2024-09-15T21:00:56.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8087000000000001e-05 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167355+0000", 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202045846 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167378+0000", 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2444999999999999e-05 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.936360207, 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:00:56.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:00:56.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:00:56.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.935550466999999, 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.970 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.835618762999999, 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:00:56.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:00:56.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.815942732000003, 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:00:56.974 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:00:56.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=6, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.531834867999997, 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19803510199999999, 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379681+0000", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379683+0000", 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.917e-06 2024-09-15T21:00:56.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379693+0000", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.01e-05 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379722+0000", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8623e-05 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.379740+0000", 2024-09-15T21:00:56.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8677000000000001e-05 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577690+0000", 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19795005800000001 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.577716+0000", 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5726999999999999e-05 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.524501016999999, 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.327077951, 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=18, reqid=client.4641.0:143, at_version=33'4, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:00:56.989 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.322692650999997, 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20013446600000001, 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588823+0000", 2024-09-15T21:00:56.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588825+0000", 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588950+0000", 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124919 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588978+0000", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8500000000000002e-05 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.588993+0000", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5237e-05 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785673+0000", 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19668001800000001 2024-09-15T21:00:56.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785700+0000", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7112999999999999e-05 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788921+0000", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0032205490000000001 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.788958+0000", 2024-09-15T21:00:56.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6619999999999998e-05 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.243842143000002, 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:00:56.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:00:56.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.551645674, 2024-09-15T21:00:56.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:00:56.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:00:57.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:00:57.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:00:57.002 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:57.003 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:56.944+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:57.003 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:56.944+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:00:57.004 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:00:57.005 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:00:57.005 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:00:57.005 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.939+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.943+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:56.943+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:57.006 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:56.999+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:57.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:00:57.045 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.040+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:00:57.048 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.044+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,6,7 to read 0,1,4,5 2024-09-15T21:00:57.104 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.100+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:57.167 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:57.168 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:57.177 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:00:57.178 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:00:57.178 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:00:57.178 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:00:57.184 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:57.184 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:57.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:00:57.200 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:57.201 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:57.217 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:57.217 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:57.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:57.236 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:57.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:57.235+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:57.254 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:57.272 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:57.272 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:57.293 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:57.293 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:57.306 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:57.299+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:57.313 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:57.313 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:57.319 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:57.319 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:57.334 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:00:57.335 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:57.354 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.348+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:57.354 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.352+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:00:57.358 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:57.382 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:00:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:00:57.447 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:57.444+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:57.454 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.452+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:00:57.454 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:57.452+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:00:57.460 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:57.460 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:00:57.461 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:00:57.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-15T21:00:57.502 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:57.502 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:57.508 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:57.503+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:57.516 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:00:57.608 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:57.604+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:57.645 INFO:tasks.rados.rados.0.smithi027.stdout:201: done (3 left) 2024-09-15T21:00:57.645 INFO:tasks.rados.rados.0.smithi027.stdout:202: read oid 46 snap -1 2024-09-15T21:00:57.645 INFO:tasks.rados.rados.0.smithi027.stdout:202: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:00:57.645 INFO:tasks.rados.rados.0.smithi027.stdout:203: read oid 3 snap -1 2024-09-15T21:00:57.645 INFO:tasks.rados.rados.0.smithi027.stdout:203: expect (ObjNum 81 snap 13 seq_num 81) 2024-09-15T21:00:57.646 INFO:tasks.rados.rados.0.smithi027.stdout:204: snap_remove snap 12 2024-09-15T21:00:57.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:00:57.691 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:57.692 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:57.706 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:00:57.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:57.703+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:57.801 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-15T21:00:57.796+0000 7ff595055640 -1 mon.a@0(leader).osd e66 definitely_dead 0 2024-09-15T21:00:57.810 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:57.803+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:57.835 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:57.835 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:57.850 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:00:57.851 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.848+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:57.852 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.848+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:57.853 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.848+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:00:57.853 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.848+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:00:57.910 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:00:57.908+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:00:57.956 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.952+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:00:57.956 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.952+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:00:57.956 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.952+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:00:57.956 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:57.952+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:00:58.012 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:58.008+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:58.023 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.023 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.038 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:00:58.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:00:58.107+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:00:58.159 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.156+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:00:58.159 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.156+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:00:58.160 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.156+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:00:58.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.160+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:58.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.160+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:00:58.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.160+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:00:58.166 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.166 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.166 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.160+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:00:58.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:00:58.215 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:58.212+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:58.316 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:58.311+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:58.344 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.344 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:00:58.373 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.368+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:58.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:58.443+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:58.474 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.468+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:58.474 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.468+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,7 to read 0,1,4,5 2024-09-15T21:00:58.474 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.468+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:00:58.519 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:58.516+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.535 INFO:teuthology.orchestra.run.smithi027.stderr:marked out osd.1. 2024-09-15T21:00:58.536 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:00:58.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:00:58.547 INFO:tasks.rados.rados.0.smithi027.stdout:204: done (5 left) 2024-09-15T21:00:58.548 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 23 current snap is 17 2024-09-15T21:00:58.548 INFO:tasks.rados.rados.0.smithi027.stdout:205: seq_num 96 ranges {1933312=458752,2392064=688128,3080192=475136} 2024-09-15T21:00:58.574 INFO:tasks.rados.rados.0.smithi027.stdout:205: writing smithi02732947-23 from 1933312 to 2392064 tid 1 2024-09-15T21:00:58.577 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.572+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:00:58.577 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.572+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:00:58.580 INFO:tasks.rados.rados.0.smithi027.stdout:205: writing smithi02732947-23 from 2392064 to 3080192 tid 2 2024-09-15T21:00:58.584 INFO:tasks.rados.rados.0.smithi027.stdout:205: writing smithi02732947-23 from 3080192 to 3555328 tid 3 2024-09-15T21:00:58.585 INFO:tasks.rados.rados.0.smithi027.stdout:206: setattr oid 6 current snap is 17 2024-09-15T21:00:58.585 INFO:tasks.rados.rados.0.smithi027.stdout:200: finishing write tid 1 to smithi02732947-22 2024-09-15T21:00:58.585 INFO:tasks.rados.rados.0.smithi027.stdout:199: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-15T21:00:58.585 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:58.590 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:58.597 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:58.603 INFO:tasks.rados.rados.0.smithi027.stdout:200: finishing write tid 2 to smithi02732947-22 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: finishing write tid 3 to smithi02732947-22 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:198: finishing copy_from racing read to smithi02732947-48 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: finishing write tid 4 to smithi02732947-22 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:198: finishing copy_from to smithi02732947-48 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 48 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: finishing write tid 5 to smithi02732947-22 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: oid 22 updating version 0 to 47 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: oid 22 updating version 47 to 48 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: oid 22 version 48 is already newer than 45 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: oid 22 version 48 is already newer than 46 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 48 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:200: left oid 22 (ObjNum 95 snap 17 seq_num 95) 2024-09-15T21:00:58.604 INFO:tasks.rados.rados.0.smithi027.stdout:202: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:00:58.614 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1543474 2024-09-15T21:00:58.620 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.616+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:58.621 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 502777 2024-09-15T21:00:58.649 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 981900 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:198: done (6 left) 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:199: done (5 left) 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:200: done (4 left) 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:202: done (3 left) 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 12 current snap is 17 2024-09-15T21:00:58.659 INFO:tasks.rados.rados.0.smithi027.stdout:207: seq_num 97 ranges {0=475136,475136=442368,917504=442368,1359872=278528} 2024-09-15T21:00:58.665 INFO:tasks.rados.rados.0.smithi027.stdout:207: writing smithi02732947-12 from 0 to 475136 tid 1 2024-09-15T21:00:58.670 INFO:tasks.rados.rados.0.smithi027.stdout:207: writing smithi02732947-12 from 475136 to 917504 tid 2 2024-09-15T21:00:58.674 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.674 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.676 INFO:tasks.rados.rados.0.smithi027.stdout:207: writing smithi02732947-12 from 917504 to 1359872 tid 3 2024-09-15T21:00:58.680 INFO:tasks.rados.rados.0.smithi027.stdout:207: writing smithi02732947-12 from 1359872 to 1638400 tid 4 2024-09-15T21:00:58.681 INFO:tasks.rados.rados.0.smithi027.stdout:208: read oid 18 snap 16 2024-09-15T21:00:58.681 INFO:tasks.rados.rados.0.smithi027.stdout:208: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:00:58.681 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 42 current snap is 17 2024-09-15T21:00:58.681 INFO:tasks.rados.rados.0.smithi027.stdout:209: seq_num 98 ranges {1523712=491520,2015232=589824,2605056=557056,3162112=16384} 2024-09-15T21:00:58.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:00:58.705 INFO:tasks.rados.rados.0.smithi027.stdout:209: writing smithi02732947-42 from 1523712 to 2015232 tid 1 2024-09-15T21:00:58.711 INFO:tasks.rados.rados.0.smithi027.stdout:209: writing smithi02732947-42 from 2015232 to 2605056 tid 2 2024-09-15T21:00:58.717 INFO:tasks.rados.rados.0.smithi027.stdout:209: writing smithi02732947-42 from 2605056 to 3162112 tid 3 2024-09-15T21:00:58.717 INFO:tasks.rados.rados.0.smithi027.stdout:209: writing smithi02732947-42 from 3162112 to 3178496 tid 4 2024-09-15T21:00:58.717 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 14 current snap is 17 2024-09-15T21:00:58.717 INFO:tasks.rados.rados.0.smithi027.stdout:210: seq_num 99 ranges {229376=507904,737280=606208,1343488=557056} 2024-09-15T21:00:58.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:58.715+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:00:58.724 INFO:tasks.rados.rados.0.smithi027.stdout:210: writing smithi02732947-14 from 229376 to 737280 tid 1 2024-09-15T21:00:58.728 INFO:tasks.rados.rados.0.smithi027.stdout:210: writing smithi02732947-14 from 737280 to 1343488 tid 2 2024-09-15T21:00:58.733 INFO:tasks.rados.rados.0.smithi027.stdout:210: writing smithi02732947-14 from 1343488 to 1900544 tid 3 2024-09-15T21:00:58.733 INFO:tasks.rados.rados.0.smithi027.stdout:211: read oid 43 snap -1 2024-09-15T21:00:58.733 INFO:tasks.rados.rados.0.smithi027.stdout:211: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T21:00:58.733 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 25 current snap is 17 2024-09-15T21:00:58.733 INFO:tasks.rados.rados.0.smithi027.stdout:212: seq_num 100 ranges {1261568=524288,1785856=753664,2539520=409600} 2024-09-15T21:00:58.746 INFO:tasks.rados.rados.0.smithi027.stdout:212: writing smithi02732947-25 from 1261568 to 1785856 tid 1 2024-09-15T21:00:58.753 INFO:tasks.rados.rados.0.smithi027.stdout:212: writing smithi02732947-25 from 1785856 to 2539520 tid 2 2024-09-15T21:00:58.757 INFO:tasks.rados.rados.0.smithi027.stdout:212: writing smithi02732947-25 from 2539520 to 2949120 tid 3 2024-09-15T21:00:58.757 INFO:tasks.rados.rados.0.smithi027.stdout:213: copy_from oid 37 from oid 47 current snap is 17 2024-09-15T21:00:58.757 INFO:tasks.rados.rados.0.smithi027.stdout:214: read oid 46 snap -1 2024-09-15T21:00:58.757 INFO:tasks.rados.rados.0.smithi027.stdout:214: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:00:58.757 INFO:tasks.rados.rados.0.smithi027.stdout:215: snap_remove snap 15 2024-09-15T21:00:58.784 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.780+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:58.784 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.780+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:58.784 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.780+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:58.785 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.780+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:00:58.785 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.780+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:00:58.812 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.812 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.821 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.816+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:58.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:00:58.884 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.880+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:00:58.884 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.880+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:58.885 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.880+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:58.922 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:00:58.920+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:00:58.956 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:58.956 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:58.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:00:58.986 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.980+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:00:58.986 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.980+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:00:58.986 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.980+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:58.986 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:58.980+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:00:59.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:00:59.019+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:00:59.125 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:59.119+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:59.133 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:59.134 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:59.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:00:59.226 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:59.224+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:59.272 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:59.272 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:59.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:00:59.334 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:59.332+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:59.387 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:59.384+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:00:59.387 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:59.384+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:00:59.387 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:00:59.384+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6 to read 0,1,4,5 2024-09-15T21:00:59.403 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:00:59.410 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:59.410 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:59.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:00:59.447 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:00:59.444+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:00:59.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:59.479+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:00:59.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:59.479+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3 to read 0,1,4,5 2024-09-15T21:00:59.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:00:59.479+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:00:59.529 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:00:59.524+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:00:59.546 INFO:tasks.rados.rados.0.smithi027.stdout:215: done (11 left) 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:216: read oid 4 snap -1 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:216: expect (ObjNum 78 snap 10 seq_num 78) 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:217: read oid 2 snap -1 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:217: expect deleted 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:218: rollback oid 9 current snap is 17 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 9 to 16 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 40 current snap is 17 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:219: seq_num 101 ranges {671744=540672,1212416=507904,1720320=622592,2342912=32768} 2024-09-15T21:00:59.547 INFO:tasks.rados.rados.0.smithi027.stdout:219: writing smithi02732947-40 from 671744 to 1212416 tid 1 2024-09-15T21:00:59.551 INFO:tasks.rados.rados.0.smithi027.stdout:219: writing smithi02732947-40 from 1212416 to 1720320 tid 2 2024-09-15T21:00:59.555 INFO:tasks.rados.rados.0.smithi027.stdout:219: writing smithi02732947-40 from 1720320 to 2342912 tid 3 2024-09-15T21:00:59.556 INFO:tasks.rados.rados.0.smithi027.stdout:219: writing smithi02732947-40 from 2342912 to 2375680 tid 4 2024-09-15T21:00:59.556 INFO:tasks.rados.rados.0.smithi027.stdout:220: rollback oid 31 current snap is 17 2024-09-15T21:00:59.556 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 31 to 16 2024-09-15T21:00:59.556 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:00:59.556 INFO:tasks.rados.rados.0.smithi027.stdout:208: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:00:59.557 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 319439 2024-09-15T21:00:59.567 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:00:59.583 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 384603 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 1 to smithi02732947-42 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 2 to smithi02732947-42 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 3 to smithi02732947-42 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 4 to smithi02732947-42 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:208: done (15 left) 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:221: delete oid 45 current snap is 17 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 5 to smithi02732947-42 2024-09-15T21:00:59.593 INFO:tasks.rados.rados.0.smithi027.stdout:205: finishing write tid 1 to smithi02732947-23 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: finishing write tid 2 to smithi02732947-23 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: finishing write tid 3 to smithi02732947-23 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: finishing write tid 4 to smithi02732947-23 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:212: finishing write tid 1 to smithi02732947-25 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:212: finishing write tid 2 to smithi02732947-25 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: finishing write tid 5 to smithi02732947-23 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: oid 23 updating version 0 to 110 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: oid 23 version 110 is already newer than 109 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: oid 23 version 110 is already newer than 107 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: oid 23 version 110 is already newer than 108 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 23 v 110 (ObjNum 96 snap 17 seq_num 96) dirty exists 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:205: left oid 23 (ObjNum 96 snap 17 seq_num 96) 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:209: finishing write tid 6 to smithi02732947-42 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:209: oid 42 updating version 0 to 99 2024-09-15T21:00:59.594 INFO:tasks.rados.rados.0.smithi027.stdout:209: oid 42 updating version 99 to 100 2024-09-15T21:00:59.595 INFO:tasks.rados.rados.0.smithi027.stdout:209: oid 42 version 100 is already newer than 98 2024-09-15T21:00:59.595 INFO:tasks.rados.rados.0.smithi027.stdout:209: oid 42 updating version 100 to 101 2024-09-15T21:00:59.595 INFO:tasks.rados.rados.0.smithi027.stdout:209: oid 42 version 101 is already newer than 97 2024-09-15T21:00:59.595 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 101 (ObjNum 98 snap 17 seq_num 98) dirty exists 2024-09-15T21:00:59.595 INFO:tasks.rados.rados.0.smithi027.stdout:209: left oid 42 (ObjNum 98 snap 17 seq_num 98) 2024-09-15T21:00:59.601 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:59.601 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:59.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:00:59.630 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:00:59.624+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:00:59.676 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:00:59.676 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:00:59.689 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 1 to smithi02732947-12 2024-09-15T21:00:59.702 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:00:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:00:59.728 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:00:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:00:59.730 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:00:59.728+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:00:59.746 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:00:59.746 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:00:59.747 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:00:59.747 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:00:59.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:00:59.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:00:59.773 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:00:59.793 INFO:tasks.rados.rados.0.smithi027.stdout:212: finishing write tid 3 to smithi02732947-25 2024-09-15T21:00:59.794 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:00:59.794 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:00:59.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:00:59.818 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:00:59.831 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:59.827+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:59.842 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:00:59.842 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:00:59.858 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 1 to smithi02732947-40 2024-09-15T21:00:59.864 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:00:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:00:59.888 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:00:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:00:59.898 INFO:tasks.rados.rados.0.smithi027.stdout:218: finishing rollback tid 0 to smithi02732947-9 2024-09-15T21:00:59.898 INFO:tasks.rados.rados.0.smithi027.stdout:218: finishing rollback tid 1 to smithi02732947-9 2024-09-15T21:00:59.918 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:00:59.919 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:00:59.932 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:00:59.927+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:00:59.938 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:00:59.938 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:00:59.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:00:59.938 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:00:59.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:425 3.4s0 3:26d7dfe1:::smithi02732947-26:head [stat,delete] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.159650687999999, 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.47973196800000001, 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 425 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776033+0000", 2024-09-15T21:00:59.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7199999999999998e-06 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776044+0000", 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1513e-05 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776077+0000", 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2452e-05 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776102+0000", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.050268+0000", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.27416589899999999 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.050531+0000", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026234400000000002 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:59.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.051801+0000", 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001270394 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.150600+0000", 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.098798945999999999 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.255555+0000", 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104955364 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.255762+0000", 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020666 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.256929+0000", 2024-09-15T21:00:59.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6787515160000002, 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29960430100000002, 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256929+0000", 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256929+0000", 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256931+0000", 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8560000000000001e-06 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256940+0000", 2024-09-15T21:00:59.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1200000000000008e-06 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256962+0000", 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1676e-05 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.256981+0000", 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8923000000000002e-05 2024-09-15T21:00:59.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556454+0000", 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29947298500000002 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556534+0000", 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9740999999999994e-05 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257074+0000", 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6786072280000006, 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29961454700000001, 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257074+0000", 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257074+0000", 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257090+0000", 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6226e-05 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257096+0000", 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9229999999999999e-06 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257135+0000", 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9112000000000001e-05 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257148+0000", 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3040000000000001e-05 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556655+0000", 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29950666100000001 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556688+0000", 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3584999999999998e-05 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257132+0000", 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6785483800000005, 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29963108100000002, 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257132+0000", 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257132+0000", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257134+0000", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4169999999999999e-06 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257139+0000", 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4960000000000004e-06 2024-09-15T21:00:59.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257153+0000", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4005999999999999e-05 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257165+0000", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.164e-05 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556740+0000", 2024-09-15T21:00:59.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299575219 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.556763+0000", 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3303e-05 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6784642259999991, 2024-09-15T21:00:59.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40017266000000001, 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:00:59.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8700000000000005e-07 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257221+0000", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999999e-06 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257243+0000", 2024-09-15T21:00:59.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1948999999999999e-05 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257257+0000", 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3616e-05 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657351+0000", 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40009474 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657389+0000", 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7758e-05 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.6784084880000005, 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40168278299999999, 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257274+0000", 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5030000000000001e-06 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257280+0000", 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257300+0000", 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9361999999999999e-05 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.357394+0000", 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100094635 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657503+0000", 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30010877400000002 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.658955+0000", 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014519649999999999 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:429 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~671744] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3381308319999992, 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.49000713600000001, 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 429 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597554+0000", 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7170000000000002e-06 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599115+0000", 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015615099999999999 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599148+0000", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3219000000000002e-05 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599165+0000", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6436999999999999e-05 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.760474+0000", 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16130950299999999 2024-09-15T21:00:59.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.760726+0000", 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025151900000000002 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763264+0000", 2024-09-15T21:00:59.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025381539999999999 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858785+0000", 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095520621999999999 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:59.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.087429+0000", 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22864422300000001 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.087557+0000", 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012823199999999999 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:430 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~770048] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3254234470000004, 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67855575999999995, 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 430 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610260+0000", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611394+0000", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011336479999999999 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611427+0000", 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3259000000000001e-05 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611443+0000", 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-05 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.765717+0000", 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15427427099999999 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.766028+0000", 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031123899999999999 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768367+0000", 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023391079999999999 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858789+0000", 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090422016999999993 2024-09-15T21:00:59.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288681+0000", 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42989205899999999 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288813+0000", 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131722 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:431 3.4s0 3:22a20f07:::smithi02732947-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3242095640000002, 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.77799016399999998, 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 431 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611472+0000", 2024-09-15T21:00:59.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.176e-06 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611480+0000", 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2269999999999996e-06 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611491+0000", 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1776e-05 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611500+0000", 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1929999999999993e-06 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770200+0000", 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15870036200000001 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770513+0000", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031272599999999999 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.771421+0000", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090811600000000004 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858791+0000", 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087370619999999996 2024-09-15T21:00:59.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389309+0000", 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.53051743900000004 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389461+0000", 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015252899999999999 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:432 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3241676929999997, 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78326424900000002, 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 432 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611514+0000", 2024-09-15T21:00:59.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.004e-06 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611517+0000", 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4029999999999999e-06 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611528+0000", 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611536+0000", 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772246+0000", 2024-09-15T21:00:59.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.160710251 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772520+0000", 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027356700000000001 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390423+0000", 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61790300099999995 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390689+0000", 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026639399999999998 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.394777+0000", 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040880530000000003 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:441 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 0~1933312,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3032552810000002, 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.34202189999999999, 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 441 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632429+0000", 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5030000000000002e-06 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632443+0000", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3890000000000001e-05 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632474+0000", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279000000000003e-05 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632490+0000", 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5899000000000001e-05 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.857996+0000", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22550603899999999 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858150+0000", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153645 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.974447+0000", 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629764500000001 2024-09-15T21:00:59.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:442 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 420736~1512576,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3031620079999993, 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.34919458599999997, 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.994 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 442 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632520+0000", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1370000000000001e-06 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632528+0000", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1100000000000003e-06 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632540+0000", 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632548+0000", 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2160000000000006e-06 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859121+0000", 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22657313700000001 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859255+0000", 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013350499999999999 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.981713+0000", 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.122458388 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.998 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:443 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 0~1933312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3031181150000002, 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.45433525000000002, 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 443 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:00:59.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632564+0000", 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632568+0000", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632582+0000", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3562e-05 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.732716+0000", 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013436000000001 2024-09-15T21:01:00.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859785+0000", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.127068872 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859920+0000", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013471199999999999 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.086898+0000", 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.226978334 2024-09-15T21:01:00.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.4s0 59/57 ECSubReadReply(tid=8, attrs_read=0))", 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.074857+0000", 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8608232739999995, 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21342040500000001, 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.074857+0000", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.074857+0000", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.074859+0000", 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.356e-06 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075296+0000", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00043734600000000002 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075324+0000", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7712000000000002e-05 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075340+0000", 2024-09-15T21:01:00.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5832999999999999e-05 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288235+0000", 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21289513199999999 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288278+0000", 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3025999999999998e-05 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.4s0 59/57 ECSubReadReply(tid=9, attrs_read=0))", 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.075366+0000", 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8603143010000007, 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21303107099999999, 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075366+0000", 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075366+0000", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075367+0000", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5000000000000001e-07 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075970+0000", 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00060259400000000002 2024-09-15T21:01:00.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075989+0000", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8513e-05 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.075997+0000", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6920000000000008e-06 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288357+0000", 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21236018300000001 2024-09-15T21:01:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288398+0000", 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0139000000000003e-05 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 59/57 ECSubWriteReply(tid=6, last_complete=59'76, committed=1, applied=1))", 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.172337+0000", 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7633439329999998, 2024-09-15T21:01:00.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21666303100000001, 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172337+0000", 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172337+0000", 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172337+0000", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.13e-07 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172340+0000", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.117e-06 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172350+0000", 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4700000000000008e-06 2024-09-15T21:01:00.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.172356+0000", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.516e-06 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.388967+0000", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21661005999999999 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389000+0000", 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3355000000000002e-05 2024-09-15T21:01:00.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 59/57 ECSubWriteReply(tid=6, last_complete=59'76, committed=1, applied=1))", 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.173387+0000", 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7622939770000006, 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.215733341, 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173387+0000", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173387+0000", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173389+0000", 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4229999999999999e-06 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173401+0000", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1619e-05 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173436+0000", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5312999999999999e-05 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173452+0000", 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5835999999999999e-05 2024-09-15T21:01:00.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389087+0000", 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21563481800000001 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389120+0000", 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3331999999999999e-05 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.4s0 59/57 ECSubReadReply(tid=9, attrs_read=0))", 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.173481+0000", 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7621997500000006, 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.218151555, 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173481+0000", 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173481+0000", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173482+0000", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.174131+0000", 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000649162 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.174149+0000", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8309e-05 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.174158+0000", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1169999999999993e-06 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.391569+0000", 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21741038900000001 2024-09-15T21:01:00.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.391633+0000", 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3887999999999995e-05 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 59/57 ECSubWriteReply(tid=6, last_complete=59'76, committed=1, applied=1))", 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.173618+0000", 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.7620631049999993, 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.216635882, 2024-09-15T21:01:00.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173618+0000", 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173618+0000", 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173620+0000", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173630+0000", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9399999999999997e-06 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173658+0000", 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8566000000000001e-05 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.173672+0000", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4106e-05 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389181+0000", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21550882199999999 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390254+0000", 2024-09-15T21:01:00.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010723180000000001 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.1s2 64/28 ECSubWrite(tid=81, reqid=client.4641.0:495, at_version=64'24, trim_to=0'0, roll_forward_to=62'22))", 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:55.654406+0000", 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.2812746849999996, 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20630069400000001, 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.654406+0000", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.654406+0000", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.654407+0000", 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3e-07 2024-09-15T21:01:00.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.654462+0000", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4688999999999998e-05 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.654494+0000", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2845999999999997e-05 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.754615+0000", 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100120498 2024-09-15T21:01:00.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.754797+0000", 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018223100000000001 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.754845+0000", 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7432000000000003e-05 2024-09-15T21:01:00.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.860646+0000", 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105801666 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.860707+0000", 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0602e-05 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.029 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.030 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:00.030 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:00.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:00.033 INFO:tasks.rados.rados.0.smithi027.stdout:212: finishing write tid 4 to smithi02732947-25 2024-09-15T21:01:00.033 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 2 to smithi02732947-12 2024-09-15T21:01:00.033 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 3 to smithi02732947-12 2024-09-15T21:01:00.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:00.031+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:00.061 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 2 to smithi02732947-40 2024-09-15T21:01:00.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:00.095+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:00.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:00.095+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,4,5 2024-09-15T21:01:00.100 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:00.095+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,4,5 2024-09-15T21:01:00.135 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:00.132+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 68/53 ECSubWriteReply(tid=85, last_complete=68'11, committed=1, applied=1))", 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.099921+0000", 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.056150482000000002, 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056178957000000002, 2024-09-15T21:01:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.099921+0000", 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.099921+0000", 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.099922+0000", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7900000000000007e-07 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.099927+0000", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8210000000000001e-06 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.099937+0000", 2024-09-15T21:01:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0332e-05 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-15T21:01:00.159 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:00.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:00.201 INFO:tasks.rados.rados.0.smithi027.stdout:212: finishing write tid 5 to smithi02732947-25 2024-09-15T21:01:00.201 INFO:tasks.rados.rados.0.smithi027.stdout:212: oid 25 updating version 0 to 110 2024-09-15T21:01:00.201 INFO:tasks.rados.rados.0.smithi027.stdout:212: oid 25 version 110 is already newer than 109 2024-09-15T21:01:00.201 INFO:tasks.rados.rados.0.smithi027.stdout:212: oid 25 version 110 is already newer than 104 2024-09-15T21:01:00.202 INFO:tasks.rados.rados.0.smithi027.stdout:212: oid 25 version 110 is already newer than 103 2024-09-15T21:01:00.202 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 110 (ObjNum 100 snap 17 seq_num 100) dirty exists 2024-09-15T21:01:00.202 INFO:tasks.rados.rados.0.smithi027.stdout:212: left oid 25 (ObjNum 100 snap 17 seq_num 100) 2024-09-15T21:01:00.202 INFO:tasks.rados.rados.0.smithi027.stdout:218: finishing rollback tid 2 to smithi02732947-9 2024-09-15T21:01:00.202 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 58 (ObjNum 61 snap 3 seq_num 61) dirty exists 2024-09-15T21:01:00.236 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:00.232+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:00.262 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 3 to smithi02732947-40 2024-09-15T21:01:00.262 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 4 to smithi02732947-40 2024-09-15T21:01:00.336 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:00.336 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:00.336 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:00.337 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:00.332+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:00.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:00.364 INFO:tasks.rados.rados.0.smithi027.stdout:220: finishing rollback tid 1 to smithi02732947-31 2024-09-15T21:01:00.364 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 31 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T21:01:00.438 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:00.436+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:00.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:00.499+0000 7fde7490f640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap e after trimming was started 2024-09-15T21:01:00.516 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:00.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.93084348, 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.517 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:01:00.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.930136122999997, 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.527 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.92956298, 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.528 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:01:00.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:01:00.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.920713765999999, 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.294194122, 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592996+0000", 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.73e-06 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593005+0000", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1689999999999998e-06 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593035+0000", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9975e-05 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.693128+0000", 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100092589 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887125+0000", 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19399696999999999 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887188+0000", 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3689000000000006e-05 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.829321039, 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.538 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:01:00.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:00.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=17, last_complete=33'3, committed=1, applied=1))", 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.821495663, 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295470228, 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692214+0000", 2024-09-15T21:01:00.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6729999999999999e-06 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692223+0000", 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8899999999999996e-06 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692249+0000", 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6310999999999999e-05 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792473+0000", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100224217 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987650+0000", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19517695600000001 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987683+0000", 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2181000000000002e-05 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.546 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.723062739, 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:01:00.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.722950304000001, 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:01:00.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:01:00.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=19, last_complete=33'5, committed=1, applied=1))", 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.715515994999997, 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29016010199999998, 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798192+0000", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798193+0000", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.443e-06 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798202+0000", 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1759999999999993e-06 2024-09-15T21:01:00.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798225+0000", 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3342000000000001e-05 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.798237+0000", 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2466000000000001e-05 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088293+0000", 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29005556199999999 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088352+0000", 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9113000000000001e-05 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=17, last_complete=33'3, committed=1, applied=1))", 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:01:00.561 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.709085774999998, 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28387544599999998, 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804622+0000", 2024-09-15T21:01:00.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804624+0000", 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804635+0000", 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0769e-05 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804663+0000", 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7622000000000001e-05 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804682+0000", 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9595000000000001e-05 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088463+0000", 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28378098899999998 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.088498+0000", 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4461000000000001e-05 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.708992934000001, 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38400288500000002, 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804716+0000", 2024-09-15T21:01:00.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804722+0000", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0689999999999998e-06 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804740+0000", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7978999999999998e-05 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904872+0000", 2024-09-15T21:01:00.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100132263 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188682+0000", 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28381031600000001 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188718+0000", 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.073260935, 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29972135599999999, 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440448+0000", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.003e-06 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440453+0000", 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1660000000000002e-06 2024-09-15T21:01:00.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440474+0000", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0801999999999998e-05 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440487+0000", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3339000000000001e-05 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740144+0000", 2024-09-15T21:01:00.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29965620500000001 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740168+0000", 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4841000000000002e-05 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.073201298999997, 2024-09-15T21:01:00.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29977237099999998, 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440512+0000", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5630000000000002e-06 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440528+0000", 2024-09-15T21:01:00.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6049000000000002e-05 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440540+0000", 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1846e-05 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740241+0000", 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29970132900000002 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740279+0000", 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7931e-05 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.073152116000003, 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299787898, 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.576 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:00.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7700000000000004e-07 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440561+0000", 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440577+0000", 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6365999999999999e-05 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440591+0000", 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3985000000000001e-05 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740326+0000", 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29973503400000001 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740344+0000", 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7756000000000001e-05 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.072915514000002, 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29959435299999998, 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440794+0000", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440800+0000", 2024-09-15T21:01:00.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9869999999999996e-06 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440821+0000", 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1308e-05 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440856+0000", 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740372+0000", 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29951594300000001 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740387+0000", 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4537e-05 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.072827066000002, 2024-09-15T21:01:00.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39968706100000001, 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440882+0000", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440886+0000", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440911+0000", 2024-09-15T21:01:00.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494999999999998e-05 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440924+0000", 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3142999999999999e-05 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840527+0000", 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39960277900000002 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840568+0000", 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.072103544999997, 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39904099900000001, 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:00.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441605+0000", 2024-09-15T21:01:00.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441611+0000", 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441652+0000", 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0890999999999999e-05 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441671+0000", 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9059000000000002e-05 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840619+0000", 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39894794099999997 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840645+0000", 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6064999999999999e-05 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:498 3.8s0 3:15433346:::smithi02732947-2:head [stat,delete] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.8638816550000001, 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40308387200000001, 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 498 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.591 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649829+0000", 2024-09-15T21:01:00.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0460000000000001e-06 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649844+0000", 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4397e-05 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649883+0000", 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804999999999999e-05 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649901+0000", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8145999999999999e-05 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.730873+0000", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080971873 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.731287+0000", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000414008 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.733618+0000", 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023318610000000002 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.735452+0000", 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018332019999999999 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052787+0000", 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.31733539399999999 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052910+0000", 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012313999999999999 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:500 3.es0 3:7cf15e4b:::smithi02732947-48:head [copy-from ver 8] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8727918130000001, 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.72001296999999997, 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 500 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640919+0000", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3100000000000001e-06 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640934+0000", 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.491e-05 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640969+0000", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4686e-05 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640986+0000", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7048999999999999e-05 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641119+0000", 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013317 2024-09-15T21:01:00.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641136+0000", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6558999999999998e-05 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.845985+0000", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20484875999999999 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846249+0000", 2024-09-15T21:01:00.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026397900000000001 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.056421+0000", 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210172042 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059407+0000", 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029868300000000002 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:00.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360789+0000", 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30138199900000001 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360929+0000", 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013967600000000001 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:501 3.es0 3:7cf15e4b:::smithi02732947-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:00.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.8726810719999998, 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41837276400000001, 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 501 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:00.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641028+0000", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641035+0000", 2024-09-15T21:01:00.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641057+0000", 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1382e-05 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641079+0000", 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1733999999999999e-05 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741365+0000", 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10028688500000001 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741386+0000", 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0423000000000001e-05 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846689+0000", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105302942 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846826+0000", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137503 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.058847+0000", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21202036799999999 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:00.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059116+0000", 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026980899999999998 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059400+0000", 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028316499999999999 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:00.608 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:00.609 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:01:00.610 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 5 to smithi02732947-40 2024-09-15T21:01:00.610 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:00.603+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:00.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:00.711 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:00.708+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:00.748 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.744+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:00.748 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.744+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:00.753 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:00.753 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:00.753 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:00.753 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:00.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:00.804 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 4 to smithi02732947-12 2024-09-15T21:01:00.804 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 5 to smithi02732947-12 2024-09-15T21:01:00.808 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:00.804+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:00.808 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:00.804+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:01:00.808 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:00.804+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,7 to read 0,1,4,5 2024-09-15T21:01:00.812 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:00.808+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:00.849 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.844+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:01:00.849 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.844+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,4,5 2024-09-15T21:01:00.862 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.856+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:00.862 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.856+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:00.862 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.856+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4 to read 0,1,4,5 2024-09-15T21:01:00.863 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.856+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: finishing write tid 6 to smithi02732947-40 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: oid 40 updating version 0 to 71 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: oid 40 version 71 is already newer than 69 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: oid 40 version 71 is already newer than 70 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: oid 40 version 71 is already newer than 67 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: oid 40 version 71 is already newer than 68 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 40 v 71 (ObjNum 101 snap 17 seq_num 101) dirty exists 2024-09-15T21:01:00.863 INFO:tasks.rados.rados.0.smithi027.stdout:219: left oid 40 (ObjNum 101 snap 17 seq_num 101) 2024-09-15T21:01:00.866 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.864+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:00.867 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.864+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:01:00.867 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.864+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:01:00.867 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.864+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:00.892 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:00.892 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:00.892 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:00.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:00.909 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:00.904+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6,7 to read 0,1,4,5 2024-09-15T21:01:00.912 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:00.907+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:00.963 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:00.960+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:00.963 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:00.960+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6 to read 0,1,4,5 2024-09-15T21:01:00.967 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:00.964+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: finishing write tid 6 to smithi02732947-12 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: oid 12 updating version 0 to 114 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: oid 12 version 114 is already newer than 112 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: oid 12 version 114 is already newer than 113 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: oid 12 version 114 is already newer than 111 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: oid 12 updating version 114 to 115 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 12 v 115 (ObjNum 97 snap 17 seq_num 97) dirty exists 2024-09-15T21:01:01.010 INFO:tasks.rados.rados.0.smithi027.stdout:207: left oid 12 (ObjNum 97 snap 17 seq_num 97) 2024-09-15T21:01:01.013 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:01.008+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:01.064 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:01.060+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,4,5 2024-09-15T21:01:01.064 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:01.060+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:01:01.066 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.060+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:01.068 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.064+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:01:01.069 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.064+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:01.072 INFO:tasks.rados.rados.0.smithi027.stdout:211: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T21:01:01.072 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:01.080 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 258748 2024-09-15T21:01:01.085 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:01.085 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:01.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:01.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:01.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.816828203, 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:01:01.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:01:01.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:01:01.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.816530063999998, 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.091 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:01:01.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:01:01.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:01:01.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.816505769999999, 2024-09-15T21:01:01.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:01:01.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:01:01.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:01.101 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.609443308000003, 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:01:01.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:01:01.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.es4 32/28 ECSubWrite(tid=13, reqid=client.4641.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.534065093999999, 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32195022499999998, 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547350+0000", 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:01:01.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547377+0000", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6630000000000001e-05 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547411+0000", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3708999999999999e-05 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547429+0000", 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:01:01.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867891+0000", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32046238700000002 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867902+0000", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1385e-05 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869267+0000", 2024-09-15T21:01:01.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013649020000000001 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869298+0000", 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0997000000000002e-05 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.es4 32/28 ECSubRead(tid=14, to_read={3:770058af:::smithi02732947-19:head=0,4096,0}, subchunks={3:770058af:::smithi02732947-19:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:01:01.109 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.531672942, 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31956152100000002, 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549740+0000", 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549742+0000", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.516e-06 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549757+0000", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5248999999999999e-05 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549789+0000", 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.205e-05 2024-09-15T21:01:01.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.549807+0000", 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7870999999999999e-05 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.868926+0000", 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31911888799999999 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869302+0000", 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037594700000000001 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.956207265, 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:01.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:01:01.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:01:01.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.856042206000001, 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:01:01.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:01:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.124 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.855997536, 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:353 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.938659558000001, 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31468109900000002, 2024-09-15T21:01:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 353 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142754+0000", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142757+0000", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3560000000000001e-06 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142771+0000", 2024-09-15T21:01:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3793e-05 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142807+0000", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6093999999999997e-05 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142833+0000", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6475e-05 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143017+0000", 2024-09-15T21:01:01.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018384099999999999 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143040+0000", 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2671999999999999e-05 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.147993+0000", 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049528979999999998 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.134 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.148147+0000", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.150417+0000", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022698869999999999 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157031+0000", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0066142179999999998 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.457335+0000", 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30030378299999999 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.457435+0000", 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9841999999999997e-05 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:01.136 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.938525937000001, 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:01.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:01:01.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:01:01.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.93843455, 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:01:01.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:01:01.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:01:01.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:359 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.775083161000001, 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35050568300000001, 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.149 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 359 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306332+0000", 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8339999999999999e-06 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306628+0000", 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029618500000000002 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306645+0000", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6518e-05 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306653+0000", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1960000000000008e-06 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306721+0000", 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7945000000000001e-05 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306833+0000", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112538 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.308213+0000", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001379915 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.309315+0000", 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001102107 2024-09-15T21:01:01.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656634+0000", 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34731901900000001 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656836+0000", 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020142599999999999 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.154 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.774737389999999, 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:01:01.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:01:01.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:361 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~262144] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.674292359999999, 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35217888200000003, 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 361 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407122+0000", 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.567e-06 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407417+0000", 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029430100000000001 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407438+0000", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1401000000000001e-05 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407450+0000", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2036e-05 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409764+0000", 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023141680000000001 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409913+0000", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000148777 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.411230+0000", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00131726 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413635+0000", 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024047719999999999 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759187+0000", 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34555227100000002 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759300+0000", 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112329 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.165 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:362 3.6s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (56) in=7b] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.673941422999999, 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35221260100000001, 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 362 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.166 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407473+0000", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407479+0000", 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5899999999999996e-06 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407492+0000", 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2435999999999999e-05 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407501+0000", 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5589999999999994e-06 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412116+0000", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046143970000000001 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412250+0000", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013454100000000001 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413018+0000", 2024-09-15T21:01:01.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076778700000000003 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413954+0000", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000935691 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759576+0000", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34562273999999998 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759684+0000", 2024-09-15T21:01:01.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010788 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.673880626999999, 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.171 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:01:01.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:426 3.6s0 3:6d8f48d3:::smithi02732947-20:head [copy-from ver 70] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.825956015999999, 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32441989700000001, 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.176 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 426 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255461+0000", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255475+0000", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.419e-05 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255507+0000", 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1507999999999997e-05 2024-09-15T21:01:01.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255522+0000", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5056e-05 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255654+0000", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013243 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255940+0000", 2024-09-15T21:01:01.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028538199999999999 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.470481+0000", 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21454129299999999 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.575970+0000", 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105488665 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579610+0000", 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003640636 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579877+0000", 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000266894 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:496 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 10] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:55.649021+0000", 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.4323924159999999, 2024-09-15T21:01:01.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31335402899999998, 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 496 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649021+0000", 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649021+0000", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649021+0000", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3499999999999996e-07 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649024+0000", 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4930000000000001e-06 2024-09-15T21:01:01.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649032+0000", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.8569999999999995e-06 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649036+0000", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7439999999999998e-06 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649178+0000", 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000141853 2024-09-15T21:01:01.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649187+0000", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6850000000000007e-06 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.652889+0000", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0037022169999999998 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.653010+0000", 2024-09-15T21:01:01.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012125000000000001 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.653682+0000", 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00067169599999999995 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.655985+0000", 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023030559999999999 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.962320+0000", 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633466500000001 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.962375+0000", 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4978000000000001e-05 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:497 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:01.187 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:55.649065+0000", 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.4323478759999997, 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31387515500000002, 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 497 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649065+0000", 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649065+0000", 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649066+0000", 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649071+0000", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.361e-06 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649077+0000", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3500000000000002e-06 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649082+0000", 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.758e-06 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649218+0000", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000136272 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.649225+0000", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8669999999999996e-06 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.654337+0000", 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0051113729999999998 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.654531+0000", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019382500000000001 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.655547+0000", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010164760000000001 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.657538+0000", 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001990599 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.962734+0000", 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30519624400000001 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:55.962940+0000", 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020664500000000001 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.193 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:01.194 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 523021 2024-09-15T21:01:01.195 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:01.196 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.160+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:01:01.196 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:01.191+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:01.275 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 25 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-15T21:01:01.280 INFO:tasks.rados.rados.0.smithi027.stdout:213: finishing copy_from racing read to smithi02732947-37 2024-09-15T21:01:01.295 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:01.291+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:01.371 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:01.371 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:01.371 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:01.371 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:01.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:01.396 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:01.391+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:01.460 INFO:tasks.rados.rados.0.smithi027.stdout:213: finishing copy_from to smithi02732947-37 2024-09-15T21:01:01.460 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T21:01:01.470 INFO:tasks.rados.rados.0.smithi027.stdout:210: finishing write tid 1 to smithi02732947-14 2024-09-15T21:01:01.483 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:01.484 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:01.484 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:01.484 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:01.485 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:01.485 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.480+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:01:01.497 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:01.492+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:01.558 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:01:01.559 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:01.559 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:01.559 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:01.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:205: done (15 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:206: done (14 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:207: done (13 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:209: done (12 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:211: done (11 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:212: done (10 left) 2024-09-15T21:01:01.580 INFO:tasks.rados.rados.0.smithi027.stdout:213: done (9 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:217: done (8 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:218: done (7 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:219: done (6 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:220: done (5 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:221: done (4 left) 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 38 current snap is 17 2024-09-15T21:01:01.581 INFO:tasks.rados.rados.0.smithi027.stdout:222: seq_num 103 ranges {1540096=573440,2113536=786432,2899968=376832} 2024-09-15T21:01:01.585 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.580+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,7 to read 0,1,4,5 2024-09-15T21:01:01.586 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.584+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:01:01.598 INFO:tasks.rados.rados.0.smithi027.stdout:222: writing smithi02732947-38 from 1540096 to 2113536 tid 1 2024-09-15T21:01:01.598 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:01.591+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:01.605 INFO:tasks.rados.rados.0.smithi027.stdout:222: writing smithi02732947-38 from 2113536 to 2899968 tid 2 2024-09-15T21:01:01.608 INFO:tasks.rados.rados.0.smithi027.stdout:222: writing smithi02732947-38 from 2899968 to 3276800 tid 3 2024-09-15T21:01:01.608 INFO:tasks.rados.rados.0.smithi027.stdout:223: rmattr oid 40 current snap is 17 2024-09-15T21:01:01.609 INFO:tasks.rados.rados.0.smithi027.stdout:223: done (5 left) 2024-09-15T21:01:01.609 INFO:tasks.rados.rados.0.smithi027.stdout:224: snap_remove snap 16 2024-09-15T21:01:01.698 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:01.696+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.946275507999999, 2024-09-15T21:01:01.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.707 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.94622845, 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.946200650000002, 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.718 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.946175255999997, 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:01:01.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.640796033999997, 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.723 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.724 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:01:01.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.724 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:01:01.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:01:01.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.640235074000003, 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.733 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.540076691000003, 2024-09-15T21:01:01.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:01:01.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:01:01.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:01:01.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.539980221, 2024-09-15T21:01:01.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:01:01.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.539022791000001, 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.746 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:01:01.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.751 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.538953284999998, 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.756 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.43918558, 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.335513407000001, 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.761 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.762 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.322253635999999, 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:01:01.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.320313208999998, 2024-09-15T21:01:01.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:01:01.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:01:01.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.130659582, 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:01:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.721649153000001, 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:01:01.775 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.714127744999999, 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:01.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:01.782 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.521248135, 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:01:01.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.520245056999997, 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:01:01.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.789 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.855692252000001, 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.793 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:01.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:01.795 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.768+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:01:01.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:01.795+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:01.800 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.796+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:01.801 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.796+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:01.801 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.796+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:01:01.801 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:01.796+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:01.900 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:01.896+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=56, reqid=client.4641.0:576, at_version=69'122, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.259072362, 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.25909460699999998, 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710927+0000", 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7000000000000001e-07 2024-09-15T21:01:01.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711169+0000", 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242449 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711184+0000", 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4956999999999999e-05 2024-09-15T21:01:01.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811338+0000", 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015429200000001 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911614+0000", 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027530699999999 2024-09-15T21:01:01.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911632+0000", 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7963000000000001e-05 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=57, reqid=client.4641.0:577, at_version=69'123, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.15860553999999999, 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.15866255300000001, 2024-09-15T21:01:01.975 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811394+0000", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1400000000000001e-06 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811565+0000", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017046300000000001 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811589+0000", 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4229000000000001e-05 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811601+0000", 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2194e-05 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912520+0000", 2024-09-15T21:01:01.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100918439 2024-09-15T21:01:01.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912530+0000", 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0720000000000001e-05 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 70/57 ECSubWrite(tid=58, reqid=client.4641.0:578, at_version=70'124, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.916413+0000", 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.053585346999999998, 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.053672367999999998, 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916413+0000", 2024-09-15T21:01:01.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916413+0000", 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916415+0000", 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7579999999999999e-06 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916449+0000", 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3998999999999998e-05 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916468+0000", 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9385000000000002e-05 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:01.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.916481+0000", 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2841e-05 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 3 2024-09-15T21:01:01.985 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:01.986 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.976+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:01.987 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.976+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:01.987 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.976+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:01:01.987 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.980+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:01.987 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.980+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:01.987 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:01.980+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:01.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:02.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:02.035 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:02.032+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:02.102 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:02.100+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:02.117 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:02.117 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:02.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:02.172 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:02.173 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:02.185 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.180+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:02.190 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:02.203 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:02.199+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:02.207 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:02.208 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:02.224 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:02.245 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:02.255 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:02.255 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:02.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:02.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:02.286 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.280+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:02.289 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:02.289 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:02.311 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:02.311 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:02.311 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:02.308+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:02.334 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:02.334 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:02.357 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:02.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:02.380 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:02.388 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.384+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:01:02.388 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.384+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:02.388 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.384+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:02.390 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.388+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:01:02.391 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.388+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:02.391 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.388+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:02.394 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.388+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,7 to read 0,1,4,5 2024-09-15T21:01:02.403 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:02.404 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:02.404 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:02.399+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:02.405 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:02.406 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:02.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:02.424 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:02.420+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:02.494 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.492+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:02.505 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:02.499+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:02.524 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:02.520+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:02.572 INFO:tasks.rados.rados.0.smithi027.stdout:224: done (5 left) 2024-09-15T21:01:02.572 INFO:tasks.rados.rados.0.smithi027.stdout:225: rmattr oid 49 current snap is 17 2024-09-15T21:01:02.572 INFO:tasks.rados.rados.0.smithi027.stdout:225: done (5 left) 2024-09-15T21:01:02.572 INFO:tasks.rados.rados.0.smithi027.stdout:226: read oid 18 snap -1 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:226: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:227: rollback oid 27 current snap is 17 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:227: done (6 left) 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:228: delete oid 42 current snap is 17 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:210: finishing write tid 2 to smithi02732947-14 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:210: finishing write tid 3 to smithi02732947-14 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:210: finishing write tid 4 to smithi02732947-14 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:203: expect (ObjNum 81 snap 13 seq_num 81) 2024-09-15T21:01:02.573 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.595 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:02.595 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:02.606 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:02.600+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:02.606 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 528386 2024-09-15T21:01:02.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:02.621 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.624 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:02.620+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:02.643 INFO:tasks.rados.rados.0.smithi027.stdout:216: expect (ObjNum 78 snap 10 seq_num 78) 2024-09-15T21:01:02.643 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.676 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.682 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.676+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:02.682 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.676+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:01:02.682 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.676+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6 to read 0,1,4,5 2024-09-15T21:01:02.682 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.676+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,4,5 2024-09-15T21:01:02.683 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.680+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:02.684 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.680+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:01:02.684 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.680+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,6 to read 0,1,4,5 2024-09-15T21:01:02.684 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.680+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,4,5 2024-09-15T21:01:02.704 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:02.703+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:02.730 INFO:tasks.rados.rados.0.smithi027.stdout:222: finishing write tid 1 to smithi02732947-38 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:222: finishing write tid 2 to smithi02732947-38 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:222: finishing write tid 3 to smithi02732947-38 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:222: finishing write tid 4 to smithi02732947-38 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: finishing write tid 5 to smithi02732947-14 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: oid 14 updating version 0 to 48 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: oid 14 updating version 48 to 51 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: oid 14 version 51 is already newer than 49 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: oid 14 version 51 is already newer than 50 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 14 v 51 (ObjNum 99 snap 17 seq_num 99) dirty exists 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:210: left oid 14 (ObjNum 99 snap 17 seq_num 99) 2024-09-15T21:01:02.731 INFO:tasks.rados.rados.0.smithi027.stdout:214: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:01:02.737 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1152485 2024-09-15T21:01:02.743 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:02.750 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:02.750 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:02.766 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:02.770 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1234934 2024-09-15T21:01:02.775 INFO:tasks.rados.rados.0.smithi027.stdout:222: finishing write tid 5 to smithi02732947-38 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:222: oid 38 updating version 0 to 122 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:222: oid 38 updating version 122 to 123 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:222: oid 38 version 123 is already newer than 121 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:222: oid 38 updating version 123 to 124 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 124 (ObjNum 103 snap 17 seq_num 103) dirty exists 2024-09-15T21:01:02.776 INFO:tasks.rados.rados.0.smithi027.stdout:222: left oid 38 (ObjNum 103 snap 17 seq_num 103) 2024-09-15T21:01:02.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.784+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:02.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.784+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:02.789 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:02.784+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,4,5 2024-09-15T21:01:02.808 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:02.804+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:02.904 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:02.904 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:02.909 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:02.903+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:02.918 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:203: done (6 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:210: done (5 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:214: done (4 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:216: done (3 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:222: done (2 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:228: done (1 left) 2024-09-15T21:01:02.983 INFO:tasks.rados.rados.0.smithi027.stdout:229: rollback oid 45 current snap is 17 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:229: done (1 left) 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:230: read oid 8 snap -1 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:230: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:231: read oid 24 snap -1 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:231: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 26 current snap is 17 2024-09-15T21:01:02.984 INFO:tasks.rados.rados.0.smithi027.stdout:232: seq_num 105 ranges {229376=606208,835584=688128,1523712=475136} 2024-09-15T21:01:02.996 INFO:tasks.rados.rados.0.smithi027.stdout:232: writing smithi02732947-26 from 229376 to 835584 tid 1 2024-09-15T21:01:03.006 INFO:tasks.rados.rados.0.smithi027.stdout:232: writing smithi02732947-26 from 835584 to 1523712 tid 2 2024-09-15T21:01:03.009 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:03.004+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:03.013 INFO:tasks.rados.rados.0.smithi027.stdout:232: writing smithi02732947-26 from 1523712 to 1998848 tid 3 2024-09-15T21:01:03.013 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 34 current snap is 17 2024-09-15T21:01:03.013 INFO:tasks.rados.rados.0.smithi027.stdout:233: seq_num 106 ranges {557056=622592,1179648=442368,1622016=524288,2146304=196608} 2024-09-15T21:01:03.031 INFO:tasks.rados.rados.0.smithi027.stdout:233: writing smithi02732947-34 from 557056 to 1179648 tid 1 2024-09-15T21:01:03.040 INFO:tasks.rados.rados.0.smithi027.stdout:233: writing smithi02732947-34 from 1179648 to 1622016 tid 2 2024-09-15T21:01:03.044 INFO:tasks.rados.rados.0.smithi027.stdout:233: writing smithi02732947-34 from 1622016 to 2146304 tid 3 2024-09-15T21:01:03.046 INFO:tasks.rados.rados.0.smithi027.stdout:233: writing smithi02732947-34 from 2146304 to 2342912 tid 4 2024-09-15T21:01:03.046 INFO:tasks.rados.rados.0.smithi027.stdout:234: rollback oid 3 current snap is 17 2024-09-15T21:01:03.047 INFO:tasks.rados.rados.0.smithi027.stdout:234: done (5 left) 2024-09-15T21:01:03.047 INFO:tasks.rados.rados.0.smithi027.stdout:235: rollback oid 38 current snap is 17 2024-09-15T21:01:03.047 INFO:tasks.rados.rados.0.smithi027.stdout:235: done (5 left) 2024-09-15T21:01:03.047 INFO:tasks.rados.rados.0.smithi027.stdout:236: snap_create 2024-09-15T21:01:03.047 INFO:tasks.rados.rados.0.smithi027.stdout:226: expect (ObjNum 66 snap 4 seq_num 66) 2024-09-15T21:01:03.057 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 661323 2024-09-15T21:01:03.077 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:03.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.079+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:03.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.079+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:03.093 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.093 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.109 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:03.110 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:03.108+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:03.117 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 394453 2024-09-15T21:01:03.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.183+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:03.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.183+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:03.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.183+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:03.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.183+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:03.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.183+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:03.211 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.208+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:03.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.215+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:03.239 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.239 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:03.312 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:03.308+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:03.386 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.386 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.387 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.383+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:03.401 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:03.445 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 1 to smithi02732947-34 2024-09-15T21:01:03.445 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 2 to smithi02732947-34 2024-09-15T21:01:03.446 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:03.443+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:03.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.483+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:03.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.483+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:03.514 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:03.507+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:03.533 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.533 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:226: done (5 left) 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:236: done (4 left) 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:237: read oid 11 snap -1 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:237: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 25 current snap is 18 2024-09-15T21:01:03.545 INFO:tasks.rados.rados.0.smithi027.stdout:238: seq_num 107 ranges {2949120=638976,3588096=458752,4046848=704512} 2024-09-15T21:01:03.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:03.569 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:03.569 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:03.569 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-15T21:01:03.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-15T21:01:03.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:01:03.585 INFO:tasks.rados.rados.0.smithi027.stdout:238: writing smithi02732947-25 from 2949120 to 3588096 tid 1 2024-09-15T21:01:03.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:03.587+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:03.591 INFO:tasks.rados.rados.0.smithi027.stdout:238: writing smithi02732947-25 from 3588096 to 4046848 tid 2 2024-09-15T21:01:03.599 INFO:tasks.rados.rados.0.smithi027.stdout:238: writing smithi02732947-25 from 4046848 to 4751360 tid 3 2024-09-15T21:01:03.599 INFO:tasks.rados.rados.0.smithi027.stdout:239: snap_remove snap 17 2024-09-15T21:01:03.614 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:03.612+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:03.715 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:03.712+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:03.725 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.725 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:03.750 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.744+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:03.750 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.744+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:03.750 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.744+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:01:03.750 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.744+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:03.750 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.744+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:03.792 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.788+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:01:03.792 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.788+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:03.817 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.812+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:03.848 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:03.843+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:03.848 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:03.843+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:01:03.849 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:03.843+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6,7 to read 0,1,4,5 2024-09-15T21:01:03.853 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.848+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:03.860 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:03.856+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:03.870 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:03.870 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:03.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:03.897 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:03.892+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:03.901 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-09-15T21:01:03.895+0000 7fd9e2db8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:moved 1 / 146 (0.684932%) 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:avg 13.2727 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:stddev 2.33903 -> 2.25996 (expected baseline 3.47363) 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:min osd.2 with 7 -> 7 pgs (0.527397 -> 0.527397 * mean) 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:max osd.10 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:oload 110 2024-09-15T21:01:03.902 INFO:teuthology.orchestra.run.smithi027.stdout:max_change 3 2024-09-15T21:01:03.903 INFO:teuthology.orchestra.run.smithi027.stdout:max_change_osds 4 2024-09-15T21:01:03.903 INFO:teuthology.orchestra.run.smithi027.stdout:average_utilization 0.0005 2024-09-15T21:01:03.903 INFO:teuthology.orchestra.run.smithi027.stdout:overload_utilization 0.0005 2024-09-15T21:01:03.903 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 weight 1.0000 -> 0.8830 2024-09-15T21:01:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:no change 2024-09-15T21:01:03.918 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:03.916+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:03.922 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-15T21:01:03.922 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-15T21:01:03.949 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:03.944+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:01:03.995 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:03.992+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:03.996 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:03.992+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.015 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.015 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.019 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:04.016+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:04.029 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:04.120 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:04.116+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:04.193 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.194 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:04.221 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:04.220+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:04.259 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.256+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:04.270 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-09-15T21:01:04.267+0000 7fd9e2db8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout:moved 1 / 146 (0.684932%) 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout:avg 13.2727 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout:stddev 2.33903 -> 2.25996 (expected baseline 3.47363) 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout:min osd.2 with 7 -> 7 pgs (0.527397 -> 0.527397 * mean) 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout:max osd.10 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-15T21:01:04.271 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:oload 110 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:max_change 3 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:max_change_osds 4 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:average_utilization 0.0005 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:overload_utilization 0.0005 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 weight 1.0000 -> 0.8830 2024-09-15T21:01:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:no change 2024-09-15T21:01:04.290 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-15T21:01:04.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-15T21:01:04.322 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:04.320+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:04.339 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.340 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.355 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:04.363 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.360+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:01:04.363 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.360+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.363 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.360+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.410 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:04.403+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:04.423 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:04.420+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:04.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:04.466 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.460+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:04.466 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.460+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:01:04.466 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.460+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.466 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.464+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.476 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.476 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:04.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:04.503+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:04.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:04.503+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:01:04.524 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:04.520+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:239: done (6 left) 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:240: rollback oid 6 current snap is 18 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:240: done (6 left) 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:241: snap_create 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 3 to smithi02732947-34 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 4 to smithi02732947-34 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 5 to smithi02732947-34 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:232: finishing write tid 1 to smithi02732947-26 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:232: finishing write tid 2 to smithi02732947-26 2024-09-15T21:01:04.543 INFO:tasks.rados.rados.0.smithi027.stdout:232: finishing write tid 3 to smithi02732947-26 2024-09-15T21:01:04.544 INFO:tasks.rados.rados.0.smithi027.stdout:231: expect (ObjNum 63 snap 3 seq_num 63) 2024-09-15T21:01:04.544 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:04.568 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.560+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:04.568 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.564+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:01:04.568 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.564+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.568 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.564+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:01:04.574 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1440448 2024-09-15T21:01:04.582 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 896298 2024-09-15T21:01:04.583 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: finishing write tid 6 to smithi02732947-34 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: oid 34 updating version 0 to 7 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: oid 34 version 7 is already newer than 3 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: oid 34 version 7 is already newer than 4 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: oid 34 version 7 is already newer than 6 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: oid 34 version 7 is already newer than 5 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 34 v 7 (ObjNum 106 snap 17 seq_num 106) dirty exists 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:233: left oid 34 (ObjNum 106 snap 17 seq_num 106) 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:232: finishing write tid 4 to smithi02732947-26 2024-09-15T21:01:04.594 INFO:tasks.rados.rados.0.smithi027.stdout:237: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T21:01:04.595 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 129120 2024-09-15T21:01:04.595 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 68106 2024-09-15T21:01:04.597 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 102221 2024-09-15T21:01:04.597 INFO:tasks.rados.rados.0.smithi027.stdout:232: finishing write tid 5 to smithi02732947-26 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:232: oid 26 updating version 0 to 128 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:232: oid 26 version 128 is already newer than 125 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:232: oid 26 version 128 is already newer than 126 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:232: oid 26 version 128 is already newer than 127 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 26 v 128 (ObjNum 105 snap 17 seq_num 105) dirty exists 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:232: left oid 26 (ObjNum 105 snap 17 seq_num 105) 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:238: finishing write tid 1 to smithi02732947-25 2024-09-15T21:01:04.598 INFO:tasks.rados.rados.0.smithi027.stdout:238: finishing write tid 2 to smithi02732947-25 2024-09-15T21:01:04.599 INFO:tasks.rados.rados.0.smithi027.stdout:238: finishing write tid 3 to smithi02732947-25 2024-09-15T21:01:04.599 INFO:tasks.rados.rados.0.smithi027.stdout:238: finishing write tid 4 to smithi02732947-25 2024-09-15T21:01:04.599 INFO:tasks.rados.rados.0.smithi027.stdout:230: expect (ObjNum 79 snap 12 seq_num 79) 2024-09-15T21:01:04.599 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:04.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:04.603 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:04.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:04.607+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:01:04.622 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:04.623 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.623 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.625 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:04.620+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:04.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:04.649 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 693739 2024-09-15T21:01:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:04.660 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:238: finishing write tid 5 to smithi02732947-25 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:238: oid 25 updating version 0 to 125 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:238: oid 25 updating version 125 to 126 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:238: oid 25 updating version 126 to 128 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:238: oid 25 version 128 is already newer than 127 2024-09-15T21:01:04.669 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 128 (ObjNum 107 snap 18 seq_num 107) dirty exists 2024-09-15T21:01:04.670 INFO:tasks.rados.rados.0.smithi027.stdout:238: left oid 25 (ObjNum 107 snap 18 seq_num 107) 2024-09-15T21:01:04.673 INFO:teuthology.orchestra.run.smithi027.stdout:moved 0 / 146 (0%) 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:avg 13.2727 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:stddev 2.33903 -> 2.33903 (expected baseline 3.47363) 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:min osd.2 with 7 -> 7 pgs (0.527397 -> 0.527397 * mean) 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:max osd.10 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:oload 1000 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:max_change 0.05 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:max_change_osds 4 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:average_utilization 0.0005 2024-09-15T21:01:04.674 INFO:teuthology.orchestra.run.smithi027.stdout:overload_utilization 0.0045 2024-09-15T21:01:04.675 INFO:teuthology.orchestra.run.smithi027.stderr:no change 2024-09-15T21:01:04.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:04.678 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:04.694 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-15T21:01:04.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-15T21:01:04.699 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:04.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:04.719+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:04.738 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:04.770 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:04.771 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:04.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:04.834 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:04.832+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:04.838 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:04.860 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:04.861 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:04.908 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:04.908 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:04.909 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:04.909 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:04.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:04.928 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:04.923+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:04.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:04.964+0000 7fe63ba3c640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap e after trimming was started 2024-09-15T21:01:05.018 INFO:tasks.ceph.mgr.x.smithi066.stderr:2024-09-15T21:01:05.011+0000 7fd9e2db8640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-15T21:01:05.018 INFO:teuthology.orchestra.run.smithi027.stdout:moved 1 / 146 (0.684932%) 2024-09-15T21:01:05.018 INFO:teuthology.orchestra.run.smithi027.stdout:avg 13.2727 2024-09-15T21:01:05.018 INFO:teuthology.orchestra.run.smithi027.stdout:stddev 2.33903 -> 2.25996 (expected baseline 3.47363) 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:min osd.2 with 7 -> 7 pgs (0.527397 -> 0.527397 * mean) 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:max osd.10 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:oload 110 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:max_change 0.05 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:max_change_osds 4 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:average_utilization 0.0005 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:overload_utilization 0.0005 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 weight 1.0000 -> 0.9500 2024-09-15T21:01:05.019 INFO:teuthology.orchestra.run.smithi027.stderr:no change 2024-09-15T21:01:05.029 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:05.024+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:05.038 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-15T21:01:05.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.797126345999999, 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:01:05.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:01:05.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.795823435000003, 2024-09-15T21:01:05.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:01:05.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.103 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.795743510000001, 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.104 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:01:05.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.589259661, 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:01:05.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:01:05.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.587862215999998, 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:01:05.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:01:05.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:05.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.587822989000003, 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:05.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.385972520999999, 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:05.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.385601704999999, 2024-09-15T21:01:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:01:05.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.385432043000002, 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:05.136 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.385023853, 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:05.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.317744009000002, 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.286531107000002, 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:01:05.144 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:01:05.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.284762237000002, 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:05.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:166 3.9s0 3:9b48fc7f:::smithi02732947-22:head [copy-from ver 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.184737755999997, 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.71698077900000001, 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904687+0000", 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-15T21:01:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904696+0000", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9899999999999992e-06 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904709+0000", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2543e-05 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904717+0000", 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:01:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904853+0000", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013647299999999999 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904974+0000", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120728 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115180+0000", 2024-09-15T21:01:05.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210206536 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116911+0000", 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730517 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621588+0000", 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.504676919 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621665+0000", 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7520000000000003e-05 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:465 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.347740685, 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36382091, 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 465 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741703+0000", 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1155999999999999e-05 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741714+0000", 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1283e-05 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741755+0000", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0698000000000002e-05 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741777+0000", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1696e-05 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.798049+0000", 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056272412000000001 2024-09-15T21:01:05.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.798323+0000", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027397100000000002 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800518+0000", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002195125 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.806827+0000", 2024-09-15T21:01:05.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006308714 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.105282+0000", 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29845517799999999 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.105503+0000", 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022067699999999999 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:466 3.9s0 3:9b48fc7f:::smithi02732947-22:head [rollback e] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.347614083, 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36587998700000002, 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741810+0000", 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7e-06 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741817+0000", 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2099999999999996e-06 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741838+0000", 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0659e-05 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741854+0000", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5930000000000002e-05 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802675+0000", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060821433000000001 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802967+0000", 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029195399999999998 2024-09-15T21:01:05.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804598+0000", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016307870000000001 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809146+0000", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004548191 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107499+0000", 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298352532 2024-09-15T21:01:05.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107688+0000", 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018959099999999999 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:467 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.347541745000001, 2024-09-15T21:01:05.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36615330200000001, 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741882+0000", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3e-06 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741888+0000", 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9170000000000003e-06 2024-09-15T21:01:05.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741910+0000", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1838999999999999e-05 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741928+0000", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8016000000000002e-05 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809749+0000", 2024-09-15T21:01:05.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067821091 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809897+0000", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148312 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810785+0000", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088809800000000003 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.812435+0000", 2024-09-15T21:01:05.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016501619999999999 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107930+0000", 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295494543 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.108034+0000", 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010402400000000001 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:506 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~540672] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4075447879999992, 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.37081094999999997, 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 506 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681880+0000", 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-15T21:01:05.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682876+0000", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099578100000000005 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682908+0000", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2138999999999998e-05 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682923+0000", 2024-09-15T21:01:05.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5577e-05 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683044+0000", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121033 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683056+0000", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1732999999999999e-05 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.746713+0000", 2024-09-15T21:01:05.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.063656931999999999 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.747027+0000", 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031414700000000003 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.749307+0000", 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022794339999999999 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.841341+0000", 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.092034070999999995 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052501+0000", 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21116006500000001 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052689+0000", 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000187708 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:507 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~622592] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.689923+0000", 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3994998459999994, 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36327325700000002, 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 507 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.689923+0000", 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.689923+0000", 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.689925+0000", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3439999999999999e-06 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690858+0000", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093358299999999998 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690891+0000", 2024-09-15T21:01:05.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2379000000000003e-05 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690907+0000", 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5838999999999999e-05 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691026+0000", 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011976000000000001 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:05.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691038+0000", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.132e-05 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.750321+0000", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059283174000000001 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.750614+0000", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029263500000000002 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.752901+0000", 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022877470000000001 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.841343+0000", 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088442046999999996 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.053008+0000", 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21166476400000001 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.053196+0000", 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000187665 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:508 3.9s0 3:9b48fc7f:::smithi02732947-22:head [setxattr _header (57) in=7b] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.3984842220000004, 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.459624861, 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 508 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:05.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690939+0000", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690947+0000", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690958+0000", 2024-09-15T21:01:05.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.128e-05 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690967+0000", 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691124+0000", 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015739899999999999 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:05.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691145+0000", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942749+0000", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25160399300000003 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942917+0000", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167522 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.943821+0000", 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090415099999999996 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.945525+0000", 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017036480000000001 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150355+0000", 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20482993199999999 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150563+0000", 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020850400000000001 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.192 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:05.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:05.194 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:05.188+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:05.294 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:05.292+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:05.352 INFO:teuthology.orchestra.run.smithi027.stdout:moved 0 / 146 (0%) 2024-09-15T21:01:05.352 INFO:teuthology.orchestra.run.smithi027.stdout:avg 13.2727 2024-09-15T21:01:05.352 INFO:teuthology.orchestra.run.smithi027.stdout:stddev 2.33903 -> 2.33903 (expected baseline 3.47363) 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:min osd.2 with 7 -> 7 pgs (0.527397 -> 0.527397 * mean) 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:max osd.10 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:oload 1000 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:max_change 0.05 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:max_change_osds 4 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:average_utilization 0.0005 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stdout:overload_utilization 0.0045 2024-09-15T21:01:05.353 INFO:teuthology.orchestra.run.smithi027.stderr:no change 2024-09-15T21:01:05.369 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:05.369 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:05.369 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:05.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:05.426 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:05.424+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:05.495 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:05.492+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:230: done (6 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:231: done (5 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:232: done (4 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:233: done (3 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:237: done (2 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:238: done (1 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:241: done (0 left) 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 16 current snap is 19 2024-09-15T21:01:05.547 INFO:tasks.rados.rados.0.smithi027.stdout:242: seq_num 108 ranges {2473984=655360,3129344=606208,3735552=475136,4210688=81920} 2024-09-15T21:01:05.556 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:05.556 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:05.556 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:05.556 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:05.556 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:05.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:05.590 INFO:tasks.rados.rados.0.smithi027.stdout:242: writing smithi02732947-16 from 2473984 to 3129344 tid 1 2024-09-15T21:01:05.596 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:05.592+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:05.598 INFO:tasks.rados.rados.0.smithi027.stdout:242: writing smithi02732947-16 from 3129344 to 3735552 tid 2 2024-09-15T21:01:05.604 INFO:tasks.rados.rados.0.smithi027.stdout:242: writing smithi02732947-16 from 3735552 to 4210688 tid 3 2024-09-15T21:01:05.605 INFO:tasks.rados.rados.0.smithi027.stdout:242: writing smithi02732947-16 from 4210688 to 4292608 tid 4 2024-09-15T21:01:05.605 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 18 current snap is 19 2024-09-15T21:01:05.605 INFO:tasks.rados.rados.0.smithi027.stdout:243: seq_num 109 ranges {1409024=671744,2080768=753664,2834432=409600} 2024-09-15T21:01:05.625 INFO:tasks.rados.rados.0.smithi027.stdout:243: writing smithi02732947-18 from 1409024 to 2080768 tid 1 2024-09-15T21:01:05.631 INFO:tasks.rados.rados.0.smithi027.stdout:243: writing smithi02732947-18 from 2080768 to 2834432 tid 2 2024-09-15T21:01:05.634 INFO:tasks.rados.rados.0.smithi027.stdout:243: writing smithi02732947-18 from 2834432 to 3244032 tid 3 2024-09-15T21:01:05.634 INFO:tasks.rados.rados.0.smithi027.stdout:244: setattr oid 45 current snap is 19 2024-09-15T21:01:05.636 INFO:tasks.rados.rados.0.smithi027.stdout:245: snap_remove snap 18 2024-09-15T21:01:05.698 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:05.692+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 33/28 ECSubRead(tid=24, to_read={3:29dd4e88:::smithi02732947-38:head=0,4096,0}, subchunks={3:29dd4e88:::smithi02732947-38:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.247670993, 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20085125200000001, 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492668+0000", 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492688+0000", 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9755999999999999e-05 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492721+0000", 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2561000000000002e-05 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.592869+0000", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100148689 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693164+0000", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10029479600000001 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693518+0000", 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035428 2024-09-15T21:01:05.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.624792862, 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20249605200000001, 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115546+0000", 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5900000000000005e-07 2024-09-15T21:01:05.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115662+0000", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011552699999999999 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115676+0000", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4547e-05 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215777+0000", 2024-09-15T21:01:05.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100100516 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215874+0000", 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7094000000000002e-05 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215886+0000", 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759999999999999e-05 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.317999+0000", 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102113494 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318041+0000", 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2154999999999999e-05 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:05.751 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.367184811999998, 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.266704728000001, 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20147183599999999, 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473634+0000", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473665+0000", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473707+0000", 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1860999999999998e-05 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473724+0000", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7206000000000001e-05 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673974+0000", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20025005600000001 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673991+0000", 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6368999999999999e-05 2024-09-15T21:01:05.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675017+0000", 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001026267 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675105+0000", 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8195000000000005e-05 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=41, reqid=client.4641.0:360, at_version=48'16, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.187436974000001, 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20315024000000001, 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552903+0000", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.561e-06 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553100+0000", 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019769300000000001 2024-09-15T21:01:05.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553123+0000", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3008999999999999e-05 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653267+0000", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014326899999999 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753534+0000", 2024-09-15T21:01:05.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100267523 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753550+0000", 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5824999999999998e-05 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755996+0000", 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024464930000000001 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756051+0000", 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4867000000000002e-05 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.ds2 58/28 ECSubWrite(tid=108, reqid=client.4641.0:438, at_version=58'8, trim_to=0'0, roll_forward_to=52'4))", 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.636764+0000", 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.103574481000001, 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.14260097499999999, 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.765 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636764+0000", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636764+0000", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636765+0000", 2024-09-15T21:01:05.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6169999999999999e-06 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636839+0000", 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4196999999999995e-05 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636869+0000", 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9793000000000001e-05 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.636885+0000", 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5599000000000001e-05 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.777393+0000", 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.14050771500000001 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.777412+0000", 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9415999999999998e-05 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.779255+0000", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018427350000000001 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.779365+0000", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010990299999999999 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:05.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.976788920000001, 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30705592100000001, 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763551+0000", 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763784+0000", 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023310900000000001 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763818+0000", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3855e-05 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763835+0000", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6393e-05 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964212+0000", 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200376947 2024-09-15T21:01:05.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964244+0000", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2035000000000001e-05 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070501+0000", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10625775599999999 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070605+0000", 2024-09-15T21:01:05.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103706 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=5, reqid=client.4641.0:430, at_version=59'75, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.971514591, 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.402708554, 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768826+0000", 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000002e-06 2024-09-15T21:01:05.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769062+0000", 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581599999999999 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769156+0000", 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4553000000000001e-05 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769216+0000", 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9932000000000001e-05 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966630+0000", 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19741336600000001 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966643+0000", 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3609e-05 2024-09-15T21:01:05.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171495+0000", 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20485210600000001 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171532+0000", 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6711999999999997e-05 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.778 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.868544546000001, 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299792744, 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:05.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871820+0000", 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5873000000000001e-05 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871847+0000", 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6792e-05 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871873+0000", 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6075999999999999e-05 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068219+0000", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19634641799999999 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068236+0000", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6912999999999999e-05 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171560+0000", 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10332366 2024-09-15T21:01:05.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171586+0000", 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6438999999999999e-05 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=7, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871902+0000", 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.86843565, 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19829734800000001, 2024-09-15T21:01:05.783 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871902+0000", 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871902+0000", 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871903+0000", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871927+0000", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3379000000000001e-05 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871946+0000", 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9544000000000001e-05 2024-09-15T21:01:05.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871960+0000", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3981999999999999e-05 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068656+0000", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196696077 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070200+0000", 2024-09-15T21:01:05.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015435830000000001 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871980+0000", 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.868358052, 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19900266799999999, 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871980+0000", 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871980+0000", 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871981+0000", 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871987+0000", 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3300000000000004e-06 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872000+0000", 2024-09-15T21:01:05.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3010000000000001e-05 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872012+0000", 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1612e-05 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070763+0000", 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19875161699999999 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070983+0000", 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021946300000000001 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.868289402, 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29954117000000002, 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:05.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872055+0000", 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8370000000000002e-06 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872067+0000", 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493e-05 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872079+0000", 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1456e-05 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171223+0000", 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299144415 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171590+0000", 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036652199999999998 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=57, reqid=client.4641.0:465, at_version=60'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.93953417, 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30393555799999999, 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-07 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800860+0000", 2024-09-15T21:01:05.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5575000000000002e-05 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800883+0000", 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3116e-05 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800903+0000", 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9426e-05 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001512+0000", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20060972299999999 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001540+0000", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7674999999999998e-05 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104682+0000", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103142055 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104739+0000", 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7481000000000001e-05 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=58, reqid=client.4641.0:466, at_version=60'38, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.935671046, 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30238109299999999, 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804669+0000", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7969999999999999e-06 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804721+0000", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1820999999999997e-05 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804762+0000", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1115999999999997e-05 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804786+0000", 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4352e-05 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103533+0000", 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29874738099999998 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103546+0000", 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2768e-05 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.106961+0000", 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415094 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107048+0000", 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6763999999999998e-05 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=59, reqid=client.4641.0:467, at_version=60'39, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:05.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.929517860000001, 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29628899199999997, 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810822+0000", 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0430000000000002e-06 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810872+0000", 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9962e-05 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810913+0000", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0642000000000002e-05 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810940+0000", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6662e-05 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104296+0000", 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29335650600000002 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104312+0000", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5559000000000001e-05 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107057+0000", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027455829999999998 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107109+0000", 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2034999999999999e-05 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.807 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.840367090999999, 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20751788099999999, 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899973+0000", 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900092+0000", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118466 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900123+0000", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1149000000000003e-05 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900146+0000", 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779000000000001e-05 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900331+0000", 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000184594 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900362+0000", 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0559000000000003e-05 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107409+0000", 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20704719299999999 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107489+0000", 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0061000000000005e-05 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=16, reqid=client.4641.0:462, at_version=60'83, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:05.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.840142242000001, 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20735974700000001, 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900197+0000", 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900419+0000", 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022236900000000001 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900447+0000", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7373e-05 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900472+0000", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981999999999999e-05 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002446+0000", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101974907 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002464+0000", 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045999999999999e-05 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107496+0000", 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105032797 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107556+0000", 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259000000000003e-05 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=17, reqid=client.4641.0:463, at_version=60'84, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.839819939, 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20715383400000001, 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:05.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900520+0000", 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900569+0000", 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9608999999999999e-05 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900593+0000", 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4034999999999999e-05 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900613+0000", 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103732+0000", 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.203119718 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103749+0000", 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107564+0000", 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038144160000000002 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107672+0000", 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108294 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 60/28 ECSubWrite(tid=183, reqid=client.4641.0:448, at_version=60'44, trim_to=0'0, roll_forward_to=56'42))", 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.831740397000001, 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20201134200000001, 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910197+0000", 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015987340000000001 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910229+0000", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2623000000000003e-05 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910247+0000", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7075000000000002e-05 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104183+0000", 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19393659899999999 2024-09-15T21:01:05.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104208+0000", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499e-05 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110513+0000", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063052070000000002 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110609+0000", 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6191000000000001e-05 2024-09-15T21:01:05.825 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs7 68/59 ECSubWrite(tid=182, reqid=client.4641.0:570, at_version=0'0, trim_to=0'0, roll_forward_to=68'70))", 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4374339679999997, 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200853328, 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302906+0000", 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.95e-06 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302929+0000", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2852000000000001e-05 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302959+0000", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9836999999999999e-05 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403039+0000", 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10008046499999999 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403103+0000", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4040000000000003e-05 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403126+0000", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2694999999999999e-05 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503703+0000", 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10057682699999999 2024-09-15T21:01:05.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503757+0000", 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4661999999999999e-05 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.830 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:05.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:05.832 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:05.827+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:05.933 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:05.928+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 74/59 ECSubWrite(tid=262, reqid=client.4641.0:617, at_version=74'136, trim_to=0'0, roll_forward_to=73'130))", 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.809487+0000", 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.150559216, 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.150601492, 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:05.960 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.809487+0000", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.809487+0000", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:05.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.809487+0000", 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1699999999999998e-07 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.809504+0000", 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6847999999999999e-05 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.809516+0000", 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2019e-05 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.909630+0000", 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100113824 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-15T21:01:05.963 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:05.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:06.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:06.027+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:06.134 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:06.132+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:06.144 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:06.145 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:06.145 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:06.145 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:06.145 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:06.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:06.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:06.211+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:06.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:06.211+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:01:06.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:06.211+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:06.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:06.211+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:01:06.236 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:06.231+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:06.315 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:06.312+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:06.316 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:06.312+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:01:06.316 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:06.312+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:01:06.316 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:06.312+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:01:06.327 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:06.327 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:425 3.4s0 3:26d7dfe1:::smithi02732947-26:head [stat,delete] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.548890874000001, 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.47973196800000001, 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.328 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 425 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776030+0000", 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776033+0000", 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7199999999999998e-06 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776044+0000", 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1513e-05 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776077+0000", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2452e-05 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:49.776102+0000", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.050268+0000", 2024-09-15T21:01:06.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.27416589899999999 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.050531+0000", 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026234400000000002 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.051801+0000", 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001270394 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.150600+0000", 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.098798945999999999 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.255555+0000", 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104955364 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.255762+0000", 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020666 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.067704412000001, 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40017266000000001, 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257217+0000", 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8700000000000005e-07 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257221+0000", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999999e-06 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257243+0000", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1948999999999999e-05 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257257+0000", 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3616e-05 2024-09-15T21:01:06.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657351+0000", 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40009474 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657389+0000", 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7758e-05 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 58/57 ECSubWriteReply(tid=2, last_complete=58'72, committed=1, applied=1))", 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.067648674000001, 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40168278299999999, 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:01:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257272+0000", 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257274+0000", 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5030000000000001e-06 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257280+0000", 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5440000000000004e-06 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.257300+0000", 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9361999999999999e-05 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.357394+0000", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100094635 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.657503+0000", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30010877400000002 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.658955+0000", 2024-09-15T21:01:06.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014519649999999999 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:429 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~671744] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.727371017999999, 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.49000713600000001, 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.343 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 429 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:01:06.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597550+0000", 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.597554+0000", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7170000000000002e-06 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599115+0000", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015615099999999999 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599148+0000", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3219000000000002e-05 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.599165+0000", 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6436999999999999e-05 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.760474+0000", 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16130950299999999 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.760726+0000", 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025151900000000002 2024-09-15T21:01:06.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763264+0000", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025381539999999999 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858785+0000", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.095520621999999999 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.087429+0000", 2024-09-15T21:01:06.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22864422300000001 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.087557+0000", 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012823199999999999 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:430 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~770048] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:06.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.714663633000001, 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67855575999999995, 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 430 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:06.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610260+0000", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611394+0000", 2024-09-15T21:01:06.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011336479999999999 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611427+0000", 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3259000000000001e-05 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611443+0000", 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-05 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.765717+0000", 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15427427099999999 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.766028+0000", 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031123899999999999 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768367+0000", 2024-09-15T21:01:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023391079999999999 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858789+0000", 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090422016999999993 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288681+0000", 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42989205899999999 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288813+0000", 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131722 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:431 3.4s0 3:22a20f07:::smithi02732947-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.713449750000001, 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.77799016399999998, 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 431 2024-09-15T21:01:06.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611472+0000", 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.176e-06 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611480+0000", 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2269999999999996e-06 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611491+0000", 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1776e-05 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611500+0000", 2024-09-15T21:01:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1929999999999993e-06 2024-09-15T21:01:06.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770200+0000", 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15870036200000001 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770513+0000", 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031272599999999999 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.771421+0000", 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090811600000000004 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858791+0000", 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087370619999999996 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389309+0000", 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.53051743900000004 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389461+0000", 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015252899999999999 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:432 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.713407879, 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78326424900000002, 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 432 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611514+0000", 2024-09-15T21:01:06.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.004e-06 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611517+0000", 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4029999999999999e-06 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611528+0000", 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-15T21:01:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611536+0000", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772246+0000", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.160710251 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772520+0000", 2024-09-15T21:01:06.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027356700000000001 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390423+0000", 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61790300099999995 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390689+0000", 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026639399999999998 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.394777+0000", 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040880530000000003 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:441 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 0~1933312,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.692495467000001, 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.34202189999999999, 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.368 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 441 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632425+0000", 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632429+0000", 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5030000000000002e-06 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632443+0000", 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3890000000000001e-05 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632474+0000", 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1279000000000003e-05 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632490+0000", 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5899000000000001e-05 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.857996+0000", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22550603899999999 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858150+0000", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000153645 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.974447+0000", 2024-09-15T21:01:06.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629764500000001 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:442 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 420736~1512576,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.692402194, 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.34919458599999997, 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 442 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632519+0000", 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632520+0000", 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1370000000000001e-06 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632528+0000", 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1100000000000003e-06 2024-09-15T21:01:06.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632540+0000", 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2092999999999999e-05 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632548+0000", 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2160000000000006e-06 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859121+0000", 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.22657313700000001 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859255+0000", 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013350499999999999 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.981713+0000", 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.122458388 2024-09-15T21:01:06.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:443 3.4s0 3:2b54c2df:::smithi02732947-23:head [read 0~1933312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.692358301000001, 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.45433525000000002, 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.378 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 443 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632563+0000", 2024-09-15T21:01:06.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632564+0000", 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4e-07 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632568+0000", 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000004e-06 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.632582+0000", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3562e-05 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.732716+0000", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013436000000001 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859785+0000", 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.127068872 2024-09-15T21:01:06.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.859920+0000", 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013471199999999999 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.086898+0000", 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.226978334 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 68/57 ECSubWriteReply(tid=47, last_complete=68'113, committed=1, applied=1))", 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.403351+0000", 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9215696160000002, 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30045867799999998, 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403351+0000", 2024-09-15T21:01:06.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403351+0000", 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403353+0000", 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3850000000000001e-06 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403362+0000", 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403378+0000", 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6147999999999999e-05 2024-09-15T21:01:06.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403388+0000", 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2590000000000006e-06 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.703774+0000", 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30038651900000002 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.703810+0000", 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5781999999999997e-05 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 68/57 ECSubWriteReply(tid=48, last_complete=68'114, committed=1, applied=1))", 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.403411+0000", 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.9215103540000005, 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30051821000000001, 2024-09-15T21:01:06.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403411+0000", 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403411+0000", 2024-09-15T21:01:06.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403411+0000", 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7599999999999998e-07 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403414+0000", 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.633e-06 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403428+0000", 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.486e-05 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503555+0000", 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100126135 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.703890+0000", 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20033538000000001 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.703929+0000", 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8825999999999997e-05 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 68/57 ECSubWriteReply(tid=49, last_complete=68'115, committed=1, applied=1))", 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.503596+0000", 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8213252180000001, 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30056465900000001, 2024-09-15T21:01:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503596+0000", 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503596+0000", 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503597+0000", 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503604+0000", 2024-09-15T21:01:06.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7170000000000008e-06 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503624+0000", 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9966000000000002e-05 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503635+0000", 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.083e-05 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.804125+0000", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.300489691 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.804160+0000", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5542e-05 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 68/57 ECSubWriteReply(tid=50, last_complete=68'117, committed=1, applied=1))", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.503660+0000", 2024-09-15T21:01:06.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.8212606869999997, 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30057011, 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503660+0000", 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503660+0000", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503661+0000", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503665+0000", 2024-09-15T21:01:06.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8870000000000002e-06 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503680+0000", 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.522e-05 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.603754+0000", 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100073605 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.804205+0000", 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20045136799999999 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.804230+0000", 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5483e-05 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:575 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~573440] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:06.399 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.599048+0000", 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.7258731479999998, 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41859928200000002, 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 575 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.599048+0000", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.599048+0000", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.599052+0000", 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0099999999999997e-06 2024-09-15T21:01:06.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.600210+0000", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011581040000000001 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.600250+0000", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0411999999999999e-05 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.600270+0000", 2024-09-15T21:01:06.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9925999999999999e-05 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607163+0000", 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0068925089999999998 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607340+0000", 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017677200000000001 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609587+0000", 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022473329999999998 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816646+0000", 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.207059034 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.017506+0000", 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200860029 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.017647+0000", 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014115300000000001 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:576 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~786432] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.7189946349999996, 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61536784200000005, 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 576 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605928+0000", 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607006+0000", 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001077862 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607032+0000", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5885999999999999e-05 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607045+0000", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662488+0000", 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055442577999999999 2024-09-15T21:01:06.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662791+0000", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030328799999999997 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.665971+0000", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031791979999999998 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816856+0000", 2024-09-15T21:01:06.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15088531299999999 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221142+0000", 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40428599700000001 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221294+0000", 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152297 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:577 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~376832] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.7156878730000003, 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61247240599999997, 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.411 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 577 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609236+0000", 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6460000000000002e-06 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610212+0000", 2024-09-15T21:01:06.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097600200000000001 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610242+0000", 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9963000000000001e-05 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610268+0000", 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712231+0000", 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101962937 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712411+0000", 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018002200000000001 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.714107+0000", 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696896 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816859+0000", 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102751021 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221545+0000", 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40468618099999998 2024-09-15T21:01:06.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221705+0000", 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016079899999999999 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:578 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.714611584, 2024-09-15T21:01:06.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61180690699999996, 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 578 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610311+0000", 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610321+0000", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9690000000000006e-06 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610342+0000", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0375999999999999e-05 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610357+0000", 2024-09-15T21:01:06.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5237e-05 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817100+0000", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20674321900000001 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817311+0000", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021106099999999999 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818686+0000", 2024-09-15T21:01:06.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013748009999999999 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.919851+0000", 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101164695 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221947+0000", 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30209587999999998 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.222116+0000", 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016968899999999999 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 70/57 ECSubWriteReply(tid=57, last_complete=69'123, committed=1, applied=1))", 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.816919+0000", 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5080024249999999, 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30340080800000002, 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.424 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816919+0000", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816919+0000", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816921+0000", 2024-09-15T21:01:06.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1229999999999998e-06 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816932+0000", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1677e-05 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816964+0000", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1492000000000003e-05 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.917069+0000", 2024-09-15T21:01:06.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10010479899999999 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.120295+0000", 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20322607400000001 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.120319+0000", 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4643000000000001e-05 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 70/57 ECSubWriteReply(tid=57, last_complete=69'123, committed=1, applied=1))", 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.818093+0000", 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.5068278949999998, 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30215630599999999, 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818093+0000", 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818093+0000", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818094+0000", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818100+0000", 2024-09-15T21:01:06.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5759999999999996e-06 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818117+0000", 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7368999999999999e-05 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818129+0000", 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2177e-05 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.120213+0000", 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30208355799999997 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.120249+0000", 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6368999999999997e-05 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.431 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:06.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:06.433 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:06.428+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:06.534 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:06.527+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:06.550 INFO:tasks.rados.rados.0.smithi027.stdout:245: done (3 left) 2024-09-15T21:01:06.550 INFO:tasks.rados.rados.0.smithi027.stdout:246: setattr oid 20 current snap is 19 2024-09-15T21:01:06.550 INFO:tasks.rados.rados.0.smithi027.stdout:243: finishing write tid 1 to smithi02732947-18 2024-09-15T21:01:06.550 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 1 to smithi02732947-16 2024-09-15T21:01:06.550 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 2 to smithi02732947-16 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 45 v 56 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: finishing write tid 2 to smithi02732947-18 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: finishing write tid 3 to smithi02732947-18 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 3 to smithi02732947-16 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: finishing write tid 4 to smithi02732947-18 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 4 to smithi02732947-16 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 5 to smithi02732947-16 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: finishing write tid 5 to smithi02732947-18 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: oid 18 updating version 0 to 75 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: oid 18 updating version 75 to 77 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: oid 18 version 77 is already newer than 76 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: oid 18 updating version 77 to 78 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 78 (ObjNum 109 snap 19 seq_num 109) dirty exists 2024-09-15T21:01:06.551 INFO:tasks.rados.rados.0.smithi027.stdout:243: left oid 18 (ObjNum 109 snap 19 seq_num 109) 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: finishing write tid 6 to smithi02732947-16 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: oid 16 updating version 0 to 135 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: oid 16 version 135 is already newer than 132 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: oid 16 version 135 is already newer than 133 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: oid 16 version 135 is already newer than 134 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: oid 16 updating version 135 to 136 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 136 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: left oid 16 (ObjNum 108 snap 19 seq_num 108) 2024-09-15T21:01:06.552 INFO:tasks.rados.rados.0.smithi027.stdout:242: done (3 left) 2024-09-15T21:01:06.553 INFO:tasks.rados.rados.0.smithi027.stdout:243: done (2 left) 2024-09-15T21:01:06.553 INFO:tasks.rados.rados.0.smithi027.stdout:244: done (1 left) 2024-09-15T21:01:06.553 INFO:tasks.rados.rados.0.smithi027.stdout:247: snap_create 2024-09-15T21:01:06.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:01:06.607 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.0s0 74/68 [PushReplyOp(3:07da53a9:::smithi02732947-6:head)])", 2024-09-15T21:01:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:06.548948+0000", 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.058250729000000001, 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058269413999999999, 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548948+0000", 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548948+0000", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548948+0000", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2000000000000001e-07 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548952+0000", 2024-09-15T21:01:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7419999999999997e-06 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548961+0000", 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9490000000000002e-06 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.548970+0000", 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6570000000000003e-06 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s4 75/56 ECSubWrite(tid=112, reqid=client.4641.0:626, at_version=75'31, trim_to=0'0, roll_forward_to=67'30))", 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.047054634999999997, 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047103700999999998, 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7599999999999996e-07 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560375+0000", 2024-09-15T21:01:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000230781 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560392+0000", 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6484999999999999e-05 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560401+0000", 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5740000000000002e-06 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s6 75/59 ECSubWrite(tid=265, reqid=osd.5.0:265, at_version=75'138, trim_to=0'0, roll_forward_to=74'136))", 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:06.564982+0000", 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.042216498999999998, 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.042289928999999997, 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.564982+0000", 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.564982+0000", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.564982+0000", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8400000000000004e-07 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.565004+0000", 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1342e-05 2024-09-15T21:01:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.565019+0000", 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4867e-05 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.565029+0000", 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.057e-05 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.616 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:06.617 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:01:06.617 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:06.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:06.666 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:06.664+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:06.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:06.731+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:06.741 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:06.741 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:06.741 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:06.741 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:06.741 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:06.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:06.837 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:06.831+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:06.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.339421139999999, 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:01:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:01:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:01:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.338713783000003, 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:01:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:01:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:01:06.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:01:06.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.338140639999999, 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:01:06.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:01:06.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:01:06.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.329291425999998, 2024-09-15T21:01:06.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.294194122, 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592994+0000", 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592996+0000", 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.73e-06 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593005+0000", 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1689999999999998e-06 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.593035+0000", 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9975e-05 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.693128+0000", 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100092589 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887125+0000", 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19399696999999999 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887188+0000", 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3689000000000006e-05 2024-09-15T21:01:06.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.237898698999999, 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:01:06.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=17, last_complete=33'3, committed=1, applied=1))", 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.230073322999999, 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295470228, 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692212+0000", 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692214+0000", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6729999999999999e-06 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692223+0000", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8899999999999996e-06 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.692249+0000", 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6310999999999999e-05 2024-09-15T21:01:06.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.792473+0000", 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100224217 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987650+0000", 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19517695600000001 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.987683+0000", 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2181000000000002e-05 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.131640398999998, 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.958 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:01:06.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:01:06.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:01:06.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.131527964, 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:06.964 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.117570594, 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38400288500000002, 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804716+0000", 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804722+0000", 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0689999999999998e-06 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804740+0000", 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7978999999999998e-05 2024-09-15T21:01:06.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904872+0000", 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100132263 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188682+0000", 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28381031600000001 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188718+0000", 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.481838594999999, 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29972135599999999, 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440447+0000", 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440448+0000", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.003e-06 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440453+0000", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1660000000000002e-06 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440474+0000", 2024-09-15T21:01:06.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0801999999999998e-05 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440487+0000", 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3339000000000001e-05 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740144+0000", 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29965620500000001 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740168+0000", 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4841000000000002e-05 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.977 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.481778959000003, 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29977237099999998, 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440507+0000", 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440512+0000", 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5630000000000002e-06 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440528+0000", 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6049000000000002e-05 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440540+0000", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1846e-05 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740241+0000", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29970132900000002 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740279+0000", 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7931e-05 2024-09-15T21:01:06.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.481729776000002, 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299787898, 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440556+0000", 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7700000000000004e-07 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440561+0000", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440577+0000", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6365999999999999e-05 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440591+0000", 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3985000000000001e-05 2024-09-15T21:01:06.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740326+0000", 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29973503400000001 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740344+0000", 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7756000000000001e-05 2024-09-15T21:01:06.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=27, last_complete=34'8, committed=1, applied=1))", 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.481493174000001, 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29959435299999998, 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.986 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440792+0000", 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440794+0000", 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440800+0000", 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9869999999999996e-06 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440821+0000", 2024-09-15T21:01:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1308e-05 2024-09-15T21:01:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440856+0000", 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5488000000000003e-05 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740372+0000", 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29951594300000001 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.740387+0000", 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4537e-05 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.991 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.481404726000001, 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39968706100000001, 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440882+0000", 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440886+0000", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440911+0000", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494999999999998e-05 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440924+0000", 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3142999999999999e-05 2024-09-15T21:01:06.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840527+0000", 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39960277900000002 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840568+0000", 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.480681205000003, 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39904099900000001, 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441605+0000", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441611+0000", 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441652+0000", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0890999999999999e-05 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441671+0000", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9059000000000002e-05 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840619+0000", 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39894794099999997 2024-09-15T21:01:06.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840645+0000", 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6064999999999999e-05 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:498 3.8s0 3:15433346:::smithi02732947-2:head [stat,delete] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.272459315000001, 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40308387200000001, 2024-09-15T21:01:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 498 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649829+0000", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0460000000000001e-06 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649844+0000", 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4397e-05 2024-09-15T21:01:07.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649883+0000", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804999999999999e-05 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649901+0000", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8145999999999999e-05 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.730873+0000", 2024-09-15T21:01:07.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080971873 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.731287+0000", 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000414008 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.733618+0000", 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023318610000000002 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:07.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.735452+0000", 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018332019999999999 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052787+0000", 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.31733539399999999 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052910+0000", 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012313999999999999 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:500 3.es0 3:7cf15e4b:::smithi02732947-48:head [copy-from ver 8] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.281369473, 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.72001296999999997, 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 500 2024-09-15T21:01:07.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:07.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640919+0000", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3100000000000001e-06 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640934+0000", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.491e-05 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640969+0000", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4686e-05 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:07.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640986+0000", 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7048999999999999e-05 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641119+0000", 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013317 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641136+0000", 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6558999999999998e-05 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.845985+0000", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20484875999999999 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846249+0000", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026397900000000001 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.056421+0000", 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210172042 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059407+0000", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029868300000000002 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360789+0000", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30138199900000001 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360929+0000", 2024-09-15T21:01:07.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013967600000000001 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:501 3.es0 3:7cf15e4b:::smithi02732947-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.281258732, 2024-09-15T21:01:07.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41837276400000001, 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 501 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:07.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641028+0000", 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:07.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641035+0000", 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641057+0000", 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1382e-05 2024-09-15T21:01:07.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641079+0000", 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1733999999999999e-05 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741365+0000", 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10028688500000001 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:07.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741386+0000", 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0423000000000001e-05 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846689+0000", 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105302942 2024-09-15T21:01:07.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846826+0000", 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137503 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.058847+0000", 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21202036799999999 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:07.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059116+0000", 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026980899999999998 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059400+0000", 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028316499999999999 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPushReply(3.0s0 74/68 [PushReplyOp(3:0196c8aa:::smithi02732947-4:head)])", 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.734224+0000", 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.1880613580000001, 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30074911100000001, 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:07.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.734224+0000", 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.734224+0000", 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.734225+0000", 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.226e-06 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.734234+0000", 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3639999999999999e-06 2024-09-15T21:01:07.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.734252+0000", 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8195000000000001e-05 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.834346+0000", 2024-09-15T21:01:07.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100093473 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.034660+0000", 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20031480700000001 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:07.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.034973+0000", 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031304599999999998 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s4 75/56 ECSubWrite(tid=112, reqid=client.4641.0:626, at_version=75'31, trim_to=0'0, roll_forward_to=67'30))", 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.362141985, 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29556577499999998, 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:07.024 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560144+0000", 2024-09-15T21:01:07.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7599999999999996e-07 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560375+0000", 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000230781 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560392+0000", 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6484999999999999e-05 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.560401+0000", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5740000000000002e-06 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.660713+0000", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100311384 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.660725+0000", 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.855674+0000", 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19494851499999999 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.855709+0000", 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:07.028 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:07.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:07.031 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:07.023+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:07.048 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:07.064 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 20 v 71 (ObjNum 77 snap 9 seq_num 77) dirty exists 2024-09-15T21:01:07.067 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:07.085 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:07.101 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:07.125 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:07.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:07.127+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:07.143 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:07.150 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.151 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.166 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:07.166 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:07.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:07.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:07.188 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:07.207 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:07.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:07.231 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:07.228+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:07.250 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:07.273 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:07.312 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.312 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:07.371 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:07.368+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:07.433 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:07.427+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.516 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:07.534 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:07.527+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:01:07.551 INFO:tasks.rados.rados.0.smithi027.stdout:246: done (1 left) 2024-09-15T21:01:07.551 INFO:tasks.rados.rados.0.smithi027.stdout:247: done (0 left) 2024-09-15T21:01:07.551 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 33 current snap is 20 2024-09-15T21:01:07.551 INFO:tasks.rados.rados.0.smithi027.stdout:248: seq_num 110 ranges {589824=688128,1277952=507904,1785856=655360} 2024-09-15T21:01:07.561 INFO:tasks.rados.rados.0.smithi027.stdout:248: writing smithi02732947-33 from 589824 to 1277952 tid 1 2024-09-15T21:01:07.566 INFO:tasks.rados.rados.0.smithi027.stdout:248: writing smithi02732947-33 from 1277952 to 1785856 tid 2 2024-09-15T21:01:07.571 INFO:tasks.rados.rados.0.smithi027.stdout:248: writing smithi02732947-33 from 1785856 to 2441216 tid 3 2024-09-15T21:01:07.572 INFO:tasks.rados.rados.0.smithi027.stdout:249: copy_from oid 18 from oid 13 current snap is 20 2024-09-15T21:01:07.572 INFO:tasks.rados.rados.0.smithi027.stdout:250: snap_create 2024-09-15T21:01:07.635 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:07.631+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:07.644 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.644 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:07.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:07.731+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:07.768 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:07.764+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:07.768 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:07.764+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:07.768 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:07.764+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:07.784 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.784 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:07.837 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:07.832+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:07.916 INFO:tasks.rados.rados.0.smithi027.stdout:249: finishing copy_from racing read to smithi02732947-18 2024-09-15T21:01:07.927 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:07.927 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:07.940 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:07.932+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:07.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:07.985 INFO:tasks.rados.rados.0.smithi027.stdout:248: finishing write tid 1 to smithi02732947-33 2024-09-15T21:01:07.986 INFO:tasks.rados.rados.0.smithi027.stdout:248: finishing write tid 2 to smithi02732947-33 2024-09-15T21:01:08.017 INFO:tasks.rados.rados.0.smithi027.stdout:249: finishing copy_from to smithi02732947-18 2024-09-15T21:01:08.018 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 84 (ObjNum 88 snap 14 seq_num 88) dirty exists 2024-09-15T21:01:08.039 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:08.036+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:08.087 INFO:tasks.rados.rados.0.smithi027.stdout:248: finishing write tid 3 to smithi02732947-33 2024-09-15T21:01:08.087 INFO:tasks.rados.rados.0.smithi027.stdout:248: finishing write tid 4 to smithi02732947-33 2024-09-15T21:01:08.089 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.084+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:08.090 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.088+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:01:08.090 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.088+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6 to read 0,1,4,5 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: finishing write tid 5 to smithi02732947-33 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: oid 33 updating version 0 to 60 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: oid 33 version 60 is already newer than 59 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: oid 33 updating version 60 to 61 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: oid 33 version 61 is already newer than 58 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 33 v 61 (ObjNum 110 snap 20 seq_num 110) dirty exists 2024-09-15T21:01:08.091 INFO:tasks.rados.rados.0.smithi027.stdout:248: left oid 33 (ObjNum 110 snap 20 seq_num 110) 2024-09-15T21:01:08.114 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.114 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:08.140 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:08.136+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:08.241 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.236+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:08.299 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.299 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.314 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:08.363 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:08.360+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:08.439 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.440 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.442 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:08.436+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:08.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:08.542 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.540+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:08.552 INFO:tasks.rados.rados.0.smithi027.stdout:248: done (2 left) 2024-09-15T21:01:08.553 INFO:tasks.rados.rados.0.smithi027.stdout:249: done (1 left) 2024-09-15T21:01:08.553 INFO:tasks.rados.rados.0.smithi027.stdout:250: done (0 left) 2024-09-15T21:01:08.553 INFO:tasks.rados.rados.0.smithi027.stdout:251: snap_create 2024-09-15T21:01:08.627 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.627 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:08.643 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:08.640+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:08.744 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:08.740+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:08.845 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:08.840+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:08.944 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:08.944 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:08.945 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:08.940+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:08.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:09.047 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:09.044+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:09.094 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:09.095 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:09.109 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:09.154 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:09.151+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:09.237 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:09.237 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:09.249 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:09.243+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:09.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:09.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:09.351 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:09.348+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:09.422 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:09.422 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:09.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:09.487 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.483+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:09.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:09.495 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:09.514 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:09.536 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:09.554 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:09.548+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:09.554 INFO:tasks.rados.rados.0.smithi027.stdout:251: done (0 left) 2024-09-15T21:01:09.554 INFO:tasks.rados.rados.0.smithi027.stdout:252: read oid 15 snap -1 2024-09-15T21:01:09.554 INFO:tasks.rados.rados.0.smithi027.stdout:252: expect (ObjNum 86 snap 14 seq_num 86) 2024-09-15T21:01:09.554 INFO:tasks.rados.rados.0.smithi027.stdout:253: delete oid 46 current snap is 22 2024-09-15T21:01:09.555 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:09.556 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:09.557 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:09.557 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:09.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:09.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:01:09.579 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:09.579 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:09.662 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:09.659+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:09.663 INFO:tasks.rados.rados.0.smithi027.stdout:253: done (1 left) 2024-09-15T21:01:09.663 INFO:tasks.rados.rados.0.smithi027.stdout:254: delete oid 36 current snap is 22 2024-09-15T21:01:09.685 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:09.685 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:09.707 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:09.708 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:09.728 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:09.749 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:09.749 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:09.749 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:09.749 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:09.749 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:09.751 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:09.751 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:09.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:09.751+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:09.764 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:09.769 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:09.778 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:09.779 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:09.803 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:09.855 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:09.852+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:09.858 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.851+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:09.858 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.851+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4 to read 0,1,4,5 2024-09-15T21:01:09.858 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.851+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:09.861 INFO:tasks.rados.rados.0.smithi027.stdout:254: done (1 left) 2024-09-15T21:01:09.861 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 44 current snap is 22 2024-09-15T21:01:09.861 INFO:tasks.rados.rados.0.smithi027.stdout:255: seq_num 113 ranges {737280=737280,1474560=557056,2031616=606208} 2024-09-15T21:01:09.869 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.863+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:09.869 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.863+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:09.869 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.863+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:01:09.869 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.863+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:01:09.882 INFO:tasks.rados.rados.0.smithi027.stdout:255: writing smithi02732947-44 from 737280 to 1474560 tid 1 2024-09-15T21:01:09.888 INFO:tasks.rados.rados.0.smithi027.stdout:255: writing smithi02732947-44 from 1474560 to 2031616 tid 2 2024-09-15T21:01:09.894 INFO:tasks.rados.rados.0.smithi027.stdout:255: writing smithi02732947-44 from 2031616 to 2637824 tid 3 2024-09-15T21:01:09.894 INFO:tasks.rados.rados.0.smithi027.stdout:256: delete oid 17 current snap is 22 2024-09-15T21:01:09.896 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:09.896 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:09.896 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:09.896 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:121 3.5s0 3:ac8a7abe:::smithi02732947-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.730665555000002, 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82723362600000006, 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.897 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 121 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161396+0000", 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3630000000000001e-06 2024-09-15T21:01:09.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162612+0000", 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001215556 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162641+0000", 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.904e-05 2024-09-15T21:01:09.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162654+0000", 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.33e-05 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262983+0000", 2024-09-15T21:01:09.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100329088 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263223+0000", 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023989200000000001 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265887+0000", 2024-09-15T21:01:09.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002663978 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369669+0000", 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10378166899999999 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988509+0000", 2024-09-15T21:01:09.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61884030999999995 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988628+0000", 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011843 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.905 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:122 3.5s0 3:ac8a7abe:::smithi02732947-41:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.905 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:01:09.905 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.729375439999998, 2024-09-15T21:01:09.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82610852099999998, 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 122 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162684+0000", 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162685+0000", 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2500000000000001e-06 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.908 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162691+0000", 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.2900000000000002e-06 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162703+0000", 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1902999999999999e-05 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162711+0000", 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3000000000000002e-06 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473128+0000", 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31041744100000002 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.473278+0000", 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014961900000000001 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474169+0000", 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00089135400000000002 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587329+0000", 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113159912 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988729+0000", 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40140014600000001 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988793+0000", 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3305999999999993e-05 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:09.913 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.729303137000002, 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.914 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:01:09.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:01:09.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:09.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.729260771, 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.921 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:01:09.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.728869844000002, 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.932 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.728830567000003, 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.937 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.689598812, 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:01:09.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:01:09.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.688876460000003, 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.943 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:01:09.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:01:09.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:09.948 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.688859968999999, 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:01:09.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:01:09.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:01:09.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.195672608999999, 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.954 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:01:09.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:01:09.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.189231145000001, 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 45.188804742999999, 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:09.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.969 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:238 3.5s0 3:a50cee1c:::smithi02732947-42:head [copy-from ver 10 in=55b] snapc 4=[4,3] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.818154995, 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.711200465, 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 238 2024-09-15T21:01:09.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073905+0000", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073908+0000", 2024-09-15T21:01:09.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1200000000000002e-06 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073922+0000", 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4392999999999999e-05 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073954+0000", 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1439000000000002e-05 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.073974+0000", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0922999999999999e-05 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.074945+0000", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097046399999999998 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.075186+0000", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024103199999999999 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:09.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.281399+0000", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20621348 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.483210+0000", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20181083799999999 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.784991+0000", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301780257 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:30.785105+0000", 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011451900000000001 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:253 3.cs0 3:3f0b4dcc:::smithi02732947-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.767309386000001, 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.65441706499999996, 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.975 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 253 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124750+0000", 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124751+0000", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124755+0000", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5499999999999999e-06 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124767+0000", 2024-09-15T21:01:09.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2758999999999999e-05 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.124779+0000", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1209999999999999e-05 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.277194+0000", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15241518800000001 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.277449+0000", 2024-09-15T21:01:09.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025487099999999999 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776732+0000", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49928296 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.776836+0000", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010461799999999999 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:31.779167+0000", 2024-09-15T21:01:09.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023311289999999999 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:349 3.5s0 3:a5da2594:::smithi02732947-25:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.749863249000001, 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.724849717, 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.980 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 349 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142196+0000", 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142198+0000", 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4559999999999999e-06 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142206+0000", 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9130000000000003e-06 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142220+0000", 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4253999999999999e-05 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142230+0000", 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0423e-05 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142468+0000", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000237712 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142491+0000", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2476e-05 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.456194+0000", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31370373499999998 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.456355+0000", 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000160896 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.867046+0000", 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.410690852 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:350 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:09.985 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.749725833999999, 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82820832200000005, 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 350 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:09.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142335+0000", 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.717e-06 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142346+0000", 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0869999999999999e-05 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142374+0000", 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7322999999999999e-05 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142395+0000", 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1063000000000001e-05 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.988 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142587+0000", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019266899999999999 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142604+0000", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6532999999999999e-05 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459842+0000", 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31723833800000001 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459989+0000", 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014666899999999999 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.970542+0000", 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51055313999999996 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:351 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:09.990 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.749612126999999, 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.83058971800000003, 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 351 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:09.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142449+0000", 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.663e-06 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142459+0000", 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0083e-05 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142477+0000", 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7842999999999998e-05 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142492+0000", 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-05 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242885+0000", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100392596 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242896+0000", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1382999999999999e-05 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460558+0000", 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217662156 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460682+0000", 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124256 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.973037+0000", 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51235473799999998 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.995 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 29.749497505000001, 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:09.996 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:09.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:09.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:494 3.5s0 3:a48c342c:::smithi02732947-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:55.649136+0000", 2024-09-15T21:01:10.000 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 14.242923122000001, 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70843455799999999, 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 494 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649136+0000", 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649136+0000", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649137+0000", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9599999999999998e-07 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649141+0000", 2024-09-15T21:01:10.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.507e-06 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649151+0000", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.061e-05 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.649159+0000", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8259999999999993e-06 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.954378+0000", 2024-09-15T21:01:10.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.305218565 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:55.954574+0000", 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019572200000000001 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.254078+0000", 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29950418299999998 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:10.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.254342+0000", 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026396600000000003 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.357571+0000", 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10322938299999999 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:618 3.5s0 3:ad5ec547:::smithi02732947-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e74)", 2024-09-15T21:01:10.005 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:05.604902+0000", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.2871579769999997, 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71257274999999998, 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 618 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604902+0000", 2024-09-15T21:01:10.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604902+0000", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604902+0000", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1999999999999998e-07 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604906+0000", 2024-09-15T21:01:10.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604920+0000", 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3396e-05 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.604928+0000", 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3240000000000003e-06 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.008 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.710290+0000", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10536176899999999 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.710437+0000", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014726299999999999 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:06.213288+0000", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50285131199999999 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:10.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:06.213545+0000", 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025625099999999998 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:06.317474+0000", 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103929748 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.010 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:10.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:10.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.955+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:10.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.955+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:01:10.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:09.955+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:01:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:10.014 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:10.007+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:10.064 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:10.059+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,7 to read 0,1,4,5 2024-09-15T21:01:10.069 INFO:tasks.rados.rados.0.smithi027.stdout:252: expect (ObjNum 86 snap 14 seq_num 86) 2024-09-15T21:01:10.079 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 607664 2024-09-15T21:01:10.112 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:10.108+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:10.115 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 469979 2024-09-15T21:01:10.143 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 381890 2024-09-15T21:01:10.188 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:10.188 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:10.188 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:10.188 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:10.213 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:10.208+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:10.297 INFO:tasks.rados.rados.0.smithi027.stdout:255: finishing write tid 1 to smithi02732947-44 2024-09-15T21:01:10.297 INFO:tasks.rados.rados.0.smithi027.stdout:255: finishing write tid 2 to smithi02732947-44 2024-09-15T21:01:10.297 INFO:tasks.rados.rados.0.smithi027.stdout:255: finishing write tid 3 to smithi02732947-44 2024-09-15T21:01:10.314 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:10.308+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:10.373 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:10.374 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:10.374 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:01:10.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:01:10.376 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:10.376 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:10.376 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:10.376 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:10.376 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:10.435 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:10.431+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:10.503 INFO:tasks.rados.rados.0.smithi027.stdout:255: finishing write tid 4 to smithi02732947-44 2024-09-15T21:01:10.504 INFO:tasks.rados.rados.0.smithi027.stdout:252: done (2 left) 2024-09-15T21:01:10.504 INFO:tasks.rados.rados.0.smithi027.stdout:256: done (1 left) 2024-09-15T21:01:10.505 INFO:tasks.rados.rados.0.smithi027.stdout:257: snap_remove snap 21 2024-09-15T21:01:10.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:10.499+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:10.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:10.499+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:10.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:10.503+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:01:10.514 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:10.511+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:10.519 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.235638924, 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.520 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.523 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.233908432, 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:10.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:01:10.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:01:10.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.527 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.232660854000002, 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:01:10.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:01:10.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.218776750000004, 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:01:10.531 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.534 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.134894750000001, 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:10.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.538 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.032447802, 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.031528903999998, 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.543 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:01:10.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:01:10.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.029005634999997, 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:01:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:01:10.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:01:10.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.026708597000002, 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:01:10.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:10.555 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.937111819999998, 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.934199139999997, 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:10.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:01:10.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 30.052649856999999, 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.564 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:01:10.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:01:10.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/28 ECSubWrite(tid=48, reqid=client.4641.0:387, at_version=53'12, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.866416587, 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203297113, 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649673+0000", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1199999999999999e-07 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649825+0000", 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015242899999999999 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649841+0000", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5931e-05 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.749979+0000", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100138358 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750181+0000", 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020151199999999999 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750213+0000", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1763000000000001e-05 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852931+0000", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102718235 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852970+0000", 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8572999999999998e-05 2024-09-15T21:01:10.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.647528456, 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:01:10.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 19.547232371, 2024-09-15T21:01:10.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20316889599999999, 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968857+0000", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8299999999999998e-07 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968860+0000", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968883+0000", 2024-09-15T21:01:10.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2989999999999998e-05 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968894+0000", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0553e-05 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.171727+0000", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20283357299999999 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172026+0000", 2024-09-15T21:01:10.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00029805000000000001 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:502 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.876083490999999, 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30380249300000001, 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:10.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 502 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640008+0000", 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8420000000000001e-06 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640020+0000", 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2067e-05 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640045+0000", 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4406999999999999e-05 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640057+0000", 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1917e-05 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640154+0000", 2024-09-15T21:01:10.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6767000000000004e-05 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640165+0000", 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.137e-05 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841240+0000", 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20107475499999999 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841489+0000", 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024974399999999999 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.943808+0000", 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102318624 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.585 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:503 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.876009891000001, 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308825238, 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 503 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640080+0000", 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4600000000000005e-07 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640085+0000", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9250000000000003e-06 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640092+0000", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3019999999999997e-06 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640096+0000", 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1919999999999998e-06 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740388+0000", 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100292057 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740400+0000", 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1420000000000001e-05 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845387+0000", 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104987575 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845666+0000", 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000279158 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.948904+0000", 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103238063 2024-09-15T21:01:10.593 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:504 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.875961379, 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409270668, 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:10.594 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 504 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.46e-07 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640133+0000", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4739999999999999e-06 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640145+0000", 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2673e-05 2024-09-15T21:01:10.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640151+0000", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2829999999999998e-06 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740469+0000", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100317608 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740477+0000", 2024-09-15T21:01:10.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2719999999999997e-06 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.846778+0000", 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106300532 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.847035+0000", 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025722499999999998 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:57.049398+0000", 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20236315499999999 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 69/59 ECSubWrite(tid=237, reqid=osd.5.0:237, at_version=69'116, trim_to=0'0, roll_forward_to=68'112))", 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:01.053816+0000", 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.462272531, 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20230262399999999, 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.599 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.053816+0000", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.053816+0000", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.053818+0000", 2024-09-15T21:01:10.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0059999999999999e-06 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.053872+0000", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3183000000000002e-05 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.053913+0000", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1836999999999999e-05 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.154069+0000", 2024-09-15T21:01:10.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10015537200000001 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.254292+0000", 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100222769 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.254307+0000", 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5393e-05 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.256042+0000", 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001735241 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:01.256119+0000", 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6823000000000006e-05 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 74/59 ECSubWrite(tid=259, reqid=client.4641.0:614, at_version=74'133, trim_to=0'0, roll_forward_to=73'130))", 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.8142264849999998, 2024-09-15T21:01:10.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20517745600000001, 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701865+0000", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4870000000000001e-06 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702223+0000", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000357544 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702264+0000", 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1263000000000001e-05 2024-09-15T21:01:10.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802363+0000", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10009968800000001 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802532+0000", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000168856 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802561+0000", 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8552e-05 2024-09-15T21:01:10.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.906960+0000", 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104399596 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.907040+0000", 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9469999999999996e-05 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.607 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.608 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.608 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:01:10.609 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:257: done (1 left) 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:258: delete oid 48 current snap is 22 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:255: finishing write tid 5 to smithi02732947-44 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:255: oid 44 updating version 0 to 140 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:255: oid 44 updating version 140 to 143 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:255: oid 44 version 143 is already newer than 142 2024-09-15T21:01:10.610 INFO:tasks.rados.rados.0.smithi027.stdout:255: oid 44 version 143 is already newer than 141 2024-09-15T21:01:10.611 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 44 v 143 (ObjNum 113 snap 22 seq_num 113) dirty exists 2024-09-15T21:01:10.611 INFO:tasks.rados.rados.0.smithi027.stdout:255: left oid 44 (ObjNum 113 snap 22 seq_num 113) 2024-09-15T21:01:10.655 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:10.651+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 79, 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "07b66685-24ed-4325-b3d4-b0cceb7d7e6d", 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2024-09-15T20:59:43.863405+0000", 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2024-09-15T21:01:10.601461+0000", 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2024-09-15T20:59:57.421497+0000", 2024-09-15T21:01:10.713 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2024-09-15T21:00:58.531515+0000", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5804032, 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "nodeep-scrub", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 6, 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:01:10.714 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 3, 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 12, 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "squid", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T20:59:55.427902+0000", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2024-09-15T21:01:10.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2024-09-15T21:01:10.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "20", 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:01:10.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:01:10.718 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:01:10.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2024-09-15T21:01:10.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 11.760000228881836, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 11.760000228881836, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.17000000178813934, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T21:01:10.721 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 2, 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "rbd", 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T21:00:04.483565+0000", 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2024-09-15T21:01:10.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 8, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 8, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 8, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 8, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 8, 2024-09-15T21:01:10.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "24", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:01:10.724 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 2, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 24, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:01:10.725 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:01:10.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd": {} 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "score_type": "Fair distribution", 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.9900000095367432, 2024-09-15T21:01:10.727 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.9900000095367432, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.67000001668930054, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 0.99999988079071045, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "unique_pool_0", 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2024-09-15T21:00:17.994699+0000", 2024-09-15T21:01:10.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 8193, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 3, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 8, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 5, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 1, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "is_stretch_pool": false, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 16, 2024-09-15T21:01:10.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 16, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 16, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 16, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 16, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2024-09-15T21:01:10.730 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "79", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 23, 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 79, 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2024-09-15T21:01:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "lrcprofile", 2024-09-15T21:01:10.732 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 16384, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2024-09-15T21:01:10.733 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": true, 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "options": {}, 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "rados": {} 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd", 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.734 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 71, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2024-09-15T21:01:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.738 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 630056637 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/630056637", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/630056637", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/630056637", 2024-09-15T21:01:10.739 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/630056637", 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "43960b2a-b250-45e3-b725-484c0751590d", 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 0, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 0, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.740 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.741 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2797309965 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/2797309965", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/2797309965", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/2797309965", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/2797309965", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "9653f0df-bf71-4a94-b79a-498f067c8de8", 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 19, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 21, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.746 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.749 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1130579408 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6825/1130579408", 2024-09-15T21:01:10.750 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6827/1130579408", 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/1130579408", 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/1130579408", 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "9ac2adf5-f248-4838-9bd5-57d13dd53309", 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.751 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 74, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2024-09-15T21:01:10.752 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2217694359 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/2217694359", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/2217694359", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6823/2217694359", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6821/2217694359", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.756 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "72c3acd6-6e64-4853-8823-947f944b96b5", 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:01:10.757 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6800", 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6801", 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.758 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6802", 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6803", 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6806", 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6807", 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6804", 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6805", 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2520277109 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6801/2520277109", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6803/2520277109", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6807/2520277109", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6805/2520277109", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "ad64f860-44e7-45a6-bc0d-e85458d80f39", 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.762 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 70, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6816", 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6817", 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6818", 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6819", 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6822", 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6823", 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6820", 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6821", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1665320626 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6817/1665320626", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6819/1665320626", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6823/1665320626", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6821/1665320626", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.767 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "4701a955-3643-4210-800d-7a86ba5155bc", 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 58, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.768 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6808", 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6809", 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6810", 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6811", 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6814", 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6815", 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6812", 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6813", 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1563776909 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6809/1563776909", 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6811/1563776909", 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6815/1563776909", 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6813/1563776909", 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2024-09-15T21:01:10.773 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "abc724fc-25c2-4b2e-b4ba-16b61dbaf89d", 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 28, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6824", 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6825", 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6826", 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6827", 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.776 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6830", 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6831", 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6828", 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.66:6829", 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3067780245 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.66:6825/3067780245", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.66:6827/3067780245", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.66:6831/3067780245", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.66:6829/3067780245", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 8, 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "b114589d-5020-4c1c-ad76-f07231e58d47", 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.779 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 72, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6824", 2024-09-15T21:01:10.780 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6825", 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6826", 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6827", 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6830", 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6831", 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6828", 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6829", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 437424413 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6825/437424413", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6827/437424413", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6831/437424413", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6829/437424413", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 9, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "6e4c938a-4505-41f5-90dc-416c6ceb868a", 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 17, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 0, 2024-09-15T21:01:10.785 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6800", 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6801", 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6802", 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6803", 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6806", 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6807", 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6804", 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6805", 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 591511703 2024-09-15T21:01:10.789 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6801/591511703", 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6803/591511703", 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6807/591511703", 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6805/591511703", 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.790 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 10, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "fa725c67-48fe-45ac-bcfb-6cf3fddea7fe", 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 57, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.791 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6808", 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6809", 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.792 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6810", 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6811", 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6814", 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6815", 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6812", 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6813", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2398207120 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6809/2398207120", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6811/2398207120", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6815/2398207120", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6813/2398207120", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 11, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "c6fcefe9-5105-45bf-ac2d-ca8eb186316c", 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 18, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 76, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2024-09-15T21:01:10.798 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6816", 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6817", 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.799 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6818", 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6819", 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6822", 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6823", 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6820", 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.92:6821", 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2517738961 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.802 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.92:6817/2517738961", 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.92:6819/2517738961", 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.92:6823/2517738961", 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.92:6821/2517738961", 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.803 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:01:03.908437+0000", 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.804 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 24621, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:00:54.445472+0000", 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.389412+0000", 2024-09-15T21:01:10.805 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.966270+0000", 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2024-09-15T21:01:10.806 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:00:55.983180+0000", 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.807 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:01:06.653717+0000", 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:01:07.037212+0000", 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:54.668403+0000", 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 8, 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.809 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:01:01.773094+0000", 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 9, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.810 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:01:00.521472+0000", 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 10, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T21:00:55.547591+0000", 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 11, 2024-09-15T21:01:10.811 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540719139924082687, 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2024-09-15T20:59:53.847328+0000", 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2024-09-15T21:01:10.812 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "lrcprofile": { 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-device-class": "", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-num-failure-domains": "0", 2024-09-15T21:01:10.813 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-osds-per-failure-domain": "0", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-root": "default", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "4", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "l": "3", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "2", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "lrcprofile", 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "lrc" 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [ 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.814 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 14, 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 16, 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 5 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 23, 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [ 2024-09-15T21:01:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 3, 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "snaps": [ 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "begin": 23, 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "length": 1 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2024-09-15T21:01:10.816 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:10.817 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s5 79/74 ECSubWrite(tid=234, reqid=osd.3.0:234, at_version=79'59, trim_to=0'0, roll_forward_to=78'57))", 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.709940+0000", 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.079899930999999993, 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.079945300999999996, 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.819 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709940+0000", 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709940+0000", 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709941+0000", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.01e-07 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709969+0000", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8614000000000001e-05 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709984+0000", 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4412e-05 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.709996+0000", 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1807000000000001e-05 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDScrubReserve(1.0,GRANT,e:79,from:11,reservation_nonce:0)", 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.777654+0000", 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.012185866, 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.012303201999999999, 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:10.822 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777654+0000", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777654+0000", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:10.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777657+0000", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777668+0000", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0624e-05 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777705+0000", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7469999999999999e-05 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.777726+0000", 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1282999999999999e-05 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 2 2024-09-15T21:01:10.825 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.826 INFO:tasks.thrashosds.thrasher:j is {'epoch': 79, 'fsid': '07b66685-24ed-4325-b3d4-b0cceb7d7e6d', 'created': '2024-09-15T20:59:43.863405+0000', 'modified': '2024-09-15T21:01:10.601461+0000', 'last_up_change': '2024-09-15T20:59:57.421497+0000', 'last_in_change': '2024-09-15T21:00:58.531515+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 12, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T20:59:55.427902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 11.760000228881836, 'score_stable': 11.760000228881836, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:00:04.483565+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-15T21:00:17.994699+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 8, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'lrcprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': True, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'd1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6808', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6809', 'nonce': 630056637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6810', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6811', 'nonce': 630056637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6814', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6815', 'nonce': 630056637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6812', 'nonce': 630056637}, {'type': 'v1', 'addr': '172.21.15.27:6813', 'nonce': 630056637}]}, 'public_addr': '172.21.15.27:6809/630056637', 'cluster_addr': '172.21.15.27:6811/630056637', 'heartbeat_back_addr': '172.21.15.27:6815/630056637', 'heartbeat_front_addr': '172.21.15.27:6813/630056637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '43960b2a-b250-45e3-b725-484c0751590d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6800', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6801', 'nonce': 2797309965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6802', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6803', 'nonce': 2797309965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6806', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6807', 'nonce': 2797309965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6804', 'nonce': 2797309965}, {'type': 'v1', 'addr': '172.21.15.27:6805', 'nonce': 2797309965}]}, 'public_addr': '172.21.15.27:6801/2797309965', 'cluster_addr': '172.21.15.27:6803/2797309965', 'heartbeat_back_addr': '172.21.15.27:6807/2797309965', 'heartbeat_front_addr': '172.21.15.27:6805/2797309965', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '9653f0df-bf71-4a94-b79a-498f067c8de8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 19, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6824', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6825', 'nonce': 1130579408}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6826', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6827', 'nonce': 1130579408}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6830', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6831', 'nonce': 1130579408}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6828', 'nonce': 1130579408}, {'type': 'v1', 'addr': '172.21.15.27:6829', 'nonce': 1130579408}]}, 'public_addr': '172.21.15.27:6825/1130579408', 'cluster_addr': '172.21.15.27:6827/1130579408', 'heartbeat_back_addr': '172.21.15.27:6831/1130579408', 'heartbeat_front_addr': '172.21.15.27:6829/1130579408', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '9ac2adf5-f248-4838-9bd5-57d13dd53309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6816', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6817', 'nonce': 2217694359}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6818', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6819', 'nonce': 2217694359}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6822', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6823', 'nonce': 2217694359}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6820', 'nonce': 2217694359}, {'type': 'v1', 'addr': '172.21.15.27:6821', 'nonce': 2217694359}]}, 'public_addr': '172.21.15.27:6817/2217694359', 'cluster_addr': '172.21.15.27:6819/2217694359', 'heartbeat_back_addr': '172.21.15.27:6823/2217694359', 'heartbeat_front_addr': '172.21.15.27:6821/2217694359', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '72c3acd6-6e64-4853-8823-947f944b96b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6800', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6801', 'nonce': 2520277109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6802', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6803', 'nonce': 2520277109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6806', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6807', 'nonce': 2520277109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6804', 'nonce': 2520277109}, {'type': 'v1', 'addr': '172.21.15.66:6805', 'nonce': 2520277109}]}, 'public_addr': '172.21.15.66:6801/2520277109', 'cluster_addr': '172.21.15.66:6803/2520277109', 'heartbeat_back_addr': '172.21.15.66:6807/2520277109', 'heartbeat_front_addr': '172.21.15.66:6805/2520277109', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ad64f860-44e7-45a6-bc0d-e85458d80f39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 70, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6816', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6817', 'nonce': 1665320626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6818', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6819', 'nonce': 1665320626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6822', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6823', 'nonce': 1665320626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6820', 'nonce': 1665320626}, {'type': 'v1', 'addr': '172.21.15.66:6821', 'nonce': 1665320626}]}, 'public_addr': '172.21.15.66:6817/1665320626', 'cluster_addr': '172.21.15.66:6819/1665320626', 'heartbeat_back_addr': '172.21.15.66:6823/1665320626', 'heartbeat_front_addr': '172.21.15.66:6821/1665320626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '4701a955-3643-4210-800d-7a86ba5155bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6808', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6809', 'nonce': 1563776909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6810', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6811', 'nonce': 1563776909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6814', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6815', 'nonce': 1563776909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6812', 'nonce': 1563776909}, {'type': 'v1', 'addr': '172.21.15.66:6813', 'nonce': 1563776909}]}, 'public_addr': '172.21.15.66:6809/1563776909', 'cluster_addr': '172.21.15.66:6811/1563776909', 'heartbeat_back_addr': '172.21.15.66:6815/1563776909', 'heartbeat_front_addr': '172.21.15.66:6813/1563776909', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'abc724fc-25c2-4b2e-b4ba-16b61dbaf89d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6824', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6825', 'nonce': 3067780245}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6826', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6827', 'nonce': 3067780245}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6830', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6831', 'nonce': 3067780245}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6828', 'nonce': 3067780245}, {'type': 'v1', 'addr': '172.21.15.66:6829', 'nonce': 3067780245}]}, 'public_addr': '172.21.15.66:6825/3067780245', 'cluster_addr': '172.21.15.66:6827/3067780245', 'heartbeat_back_addr': '172.21.15.66:6831/3067780245', 'heartbeat_front_addr': '172.21.15.66:6829/3067780245', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'b114589d-5020-4c1c-ad76-f07231e58d47', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6824', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6825', 'nonce': 437424413}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6826', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6827', 'nonce': 437424413}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6830', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6831', 'nonce': 437424413}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6828', 'nonce': 437424413}, {'type': 'v1', 'addr': '172.21.15.92:6829', 'nonce': 437424413}]}, 'public_addr': '172.21.15.92:6825/437424413', 'cluster_addr': '172.21.15.92:6827/437424413', 'heartbeat_back_addr': '172.21.15.92:6831/437424413', 'heartbeat_front_addr': '172.21.15.92:6829/437424413', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '6e4c938a-4505-41f5-90dc-416c6ceb868a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 17, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6800', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6801', 'nonce': 591511703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6802', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6803', 'nonce': 591511703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6806', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6807', 'nonce': 591511703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6804', 'nonce': 591511703}, {'type': 'v1', 'addr': '172.21.15.92:6805', 'nonce': 591511703}]}, 'public_addr': '172.21.15.92:6801/591511703', 'cluster_addr': '172.21.15.92:6803/591511703', 'heartbeat_back_addr': '172.21.15.92:6807/591511703', 'heartbeat_front_addr': '172.21.15.92:6805/591511703', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': 'fa725c67-48fe-45ac-bcfb-6cf3fddea7fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6808', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6809', 'nonce': 2398207120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6810', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6811', 'nonce': 2398207120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6814', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6815', 'nonce': 2398207120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6812', 'nonce': 2398207120}, {'type': 'v1', 'addr': '172.21.15.92:6813', 'nonce': 2398207120}]}, 'public_addr': '172.21.15.92:6809/2398207120', 'cluster_addr': '172.21.15.92:6811/2398207120', 'heartbeat_back_addr': '172.21.15.92:6815/2398207120', 'heartbeat_front_addr': '172.21.15.92:6813/2398207120', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'c6fcefe9-5105-45bf-ac2d-ca8eb186316c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6816', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6817', 'nonce': 2517738961}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6818', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6819', 'nonce': 2517738961}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6822', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6823', 'nonce': 2517738961}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6820', 'nonce': 2517738961}, {'type': 'v1', 'addr': '172.21.15.92:6821', 'nonce': 2517738961}]}, 'public_addr': '172.21.15.92:6817/2517738961', 'cluster_addr': '172.21.15.92:6819/2517738961', 'heartbeat_back_addr': '172.21.15.92:6823/2517738961', 'heartbeat_front_addr': '172.21.15.92:6821/2517738961', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:01:03.908437+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 24621, 'last_purged_snaps_scrub': '2024-09-15T21:00:54.445472+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.389412+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.966270+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:00:55.983180+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:01:06.653717+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:01:07.037212+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:54.668403+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:01:01.773094+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:01:00.521472+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:00:55.547591+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T20:59:53.847328+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'lrcprofile': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'l': '3', 'm': '2', 'name': 'lrcprofile', 'plugin': 'lrc'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 14, 'length': 1}, {'begin': 16, 'length': 5}, {'begin': 23, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:01:10.826 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T21:01:10.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:10.827 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:10.815+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:255: done (1 left) 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:258: done (0 left) 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:259: copy_from oid 36 from oid 46 current snap is 22 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:260: rollback oid 15 current snap is 22 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 15 to 20 2024-09-15T21:01:10.828 INFO:tasks.rados.rados.0.smithi027.stdout:261: setattr oid 14 current snap is 22 2024-09-15T21:01:10.831 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 50 current snap is 22 2024-09-15T21:01:10.831 INFO:tasks.rados.rados.0.smithi027.stdout:262: seq_num 116 ranges {835584=786432,1622016=458752,2080768=442368,2523136=262144} 2024-09-15T21:01:10.858 INFO:tasks.rados.rados.0.smithi027.stdout:262: writing smithi02732947-50 from 835584 to 1622016 tid 1 2024-09-15T21:01:10.866 INFO:tasks.rados.rados.0.smithi027.stdout:262: writing smithi02732947-50 from 1622016 to 2080768 tid 2 2024-09-15T21:01:10.873 INFO:tasks.rados.rados.0.smithi027.stdout:262: writing smithi02732947-50 from 2080768 to 2523136 tid 3 2024-09-15T21:01:10.877 INFO:tasks.rados.rados.0.smithi027.stdout:262: writing smithi02732947-50 from 2523136 to 2785280 tid 4 2024-09-15T21:01:10.877 INFO:tasks.rados.rados.0.smithi027.stdout:263: read oid 12 snap 20 2024-09-15T21:01:10.877 INFO:tasks.rados.rados.0.smithi027.stdout:263: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-15T21:01:10.877 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 27 current snap is 22 2024-09-15T21:01:10.877 INFO:tasks.rados.rados.0.smithi027.stdout:264: seq_num 117 ranges {442368=425984,868352=606208,1474560=540672,2015232=409600} 2024-09-15T21:01:10.888 INFO:tasks.rados.rados.0.smithi027.stdout:264: writing smithi02732947-27 from 442368 to 868352 tid 1 2024-09-15T21:01:10.894 INFO:tasks.rados.rados.0.smithi027.stdout:264: writing smithi02732947-27 from 868352 to 1474560 tid 2 2024-09-15T21:01:10.900 INFO:tasks.rados.rados.0.smithi027.stdout:264: writing smithi02732947-27 from 1474560 to 2015232 tid 3 2024-09-15T21:01:10.904 INFO:tasks.rados.rados.0.smithi027.stdout:264: writing smithi02732947-27 from 2015232 to 2424832 tid 4 2024-09-15T21:01:10.904 INFO:tasks.rados.rados.0.smithi027.stdout:265: rmattr oid 40 current snap is 22 2024-09-15T21:01:10.904 INFO:tasks.rados.rados.0.smithi027.stdout:265: done (6 left) 2024-09-15T21:01:10.904 INFO:tasks.rados.rados.0.smithi027.stdout:266: copy_from oid 3 from oid 26 current snap is 22 2024-09-15T21:01:10.905 INFO:tasks.rados.rados.0.smithi027.stdout:267: read oid 20 snap -1 2024-09-15T21:01:10.905 INFO:tasks.rados.rados.0.smithi027.stdout:267: expect (ObjNum 77 snap 9 seq_num 77) 2024-09-15T21:01:10.905 INFO:tasks.rados.rados.0.smithi027.stdout:268: rmattr oid 2 current snap is 22 2024-09-15T21:01:10.905 INFO:tasks.rados.rados.0.smithi027.stdout:268: done (8 left) 2024-09-15T21:01:10.905 INFO:tasks.rados.rados.0.smithi027.stdout:269: delete oid 44 current snap is 22 2024-09-15T21:01:10.919 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:10.915+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:10.961 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:10.961 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:10.961 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:10.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:11.017 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.012+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:11.017 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.012+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:11.017 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.012+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:11.017 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.012+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:01:11.018 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.012+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,4,5 2024-09-15T21:01:11.020 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.015+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:11.029 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.023+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:11.029 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.023+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:01:11.029 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.023+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,4,5 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:16 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.allocated (16) in=17b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.482536727999999, 2024-09-15T21:01:11.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40343498799999999, 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 16 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618250+0000", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618253+0000", 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3800000000000001e-06 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618265+0000", 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2245999999999999e-05 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.618296+0000", 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0953e-05 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.718394+0000", 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100098231 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818630+0000", 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100236266 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818771+0000", 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000141039 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.818989+0000", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000217579 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:58.820198+0000", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012090040000000001 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021536+0000", 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20133788799999999 2024-09-15T21:01:11.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021620+0000", 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4209999999999995e-05 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.021685+0000", 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5191999999999995e-05 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.111 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 71.869242469, 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40450274800000002, 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 24 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231545+0000", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231546+0000", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.066e-06 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231686+0000", 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014017200000000001 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231699+0000", 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3550000000000001e-05 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231711+0000", 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1337e-05 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235444+0000", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003732998 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235565+0000", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012092899999999999 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.235764+0000", 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019989899999999999 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333100+0000", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097335838999999993 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635764+0000", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30266414000000003 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.635925+0000", 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016088499999999999 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.636047+0000", 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121933 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 71.869057767000001, 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.117 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.122 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.663781756000006, 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:01:11.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.663456595, 2024-09-15T21:01:11.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:01:11.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:01:11.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:01:11.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.663415737999998, 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.134 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:01:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.663366991999993, 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:01:11.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.145 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.663052081000004, 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:01:11.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:01:11.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 70.662974015000003, 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:11.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:01:11.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:01:11.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 66.872518310000004, 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:01:11.157 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:01:11.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:01:11.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:01:11.162 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 66.872348267000007, 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:11.163 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:437 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~540672] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.471928945999998, 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54912561999999998, 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 437 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:11.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628861+0000", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7499999999999999e-06 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629602+0000", 2024-09-15T21:01:11.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142900000000005 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629633+0000", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0453000000000001e-05 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629649+0000", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5923e-05 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629768+0000", 2024-09-15T21:01:11.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119453 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.630003+0000", 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023483799999999999 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.632434+0000", 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024312940000000001 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.636553+0000", 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041183879999999997 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177838+0000", 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54128565900000003 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177984+0000", 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014543299999999999 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:438 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~229376] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.469360250000001, 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54706032999999998, 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 438 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631429+0000", 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2630000000000002e-06 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631860+0000", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000431261 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631886+0000", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6230000000000001e-05 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631907+0000", 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:11.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633475+0000", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015675870000000001 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633636+0000", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016115899999999999 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.635458+0000", 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018221299999999999 2024-09-15T21:01:11.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.637212+0000", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017535420000000001 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178395+0000", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54118301599999996 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178487+0000", 2024-09-15T21:01:11.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2262999999999994e-05 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:439 3.ds0 3:b220672c:::smithi02732947-15:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.468833750000002, 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.85063377100000004, 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 439 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.180 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631956+0000", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-06 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631966+0000", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0047000000000001e-05 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631987+0000", 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1253000000000001e-05 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732106+0000", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011937999999999 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179003+0000", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44689677 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179116+0000", 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112794 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179656+0000", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054039500000000005 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180543+0000", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088661899999999995 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482437+0000", 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30189397699999998 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482587+0000", 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015026300000000001 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:440 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.368636828, 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.75401734600000003, 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.184 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 440 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732153+0000", 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.717e-06 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732167+0000", 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.382e-05 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732202+0000", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5553000000000003e-05 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732225+0000", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2894000000000001e-05 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180004+0000", 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44777894899999998 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180092+0000", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7705000000000001e-05 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483149+0000", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30305710299999999 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483259+0000", 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109998 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.486168+0000", 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0029086070000000001 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.323742101000001, 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40123385299999997, 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.189 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777045+0000", 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3900000000000001e-07 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777047+0000", 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7519999999999999e-06 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777053+0000", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8749999999999997e-06 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.777057+0000", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5580000000000002e-06 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178267+0000", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40120943799999997 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.192 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178279+0000", 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1990999999999999e-05 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.270275085000002, 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34768011700000001, 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830512+0000", 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.194 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830513+0000", 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.587e-06 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830522+0000", 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.303e-06 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.195 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.830549+0000", 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6820000000000001e-05 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.930645+0000", 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100096688 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178169+0000", 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.247523936 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.196 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178192+0000", 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2782999999999999e-05 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=107, last_complete=58'7, committed=1, applied=1))", 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.267658390000001, 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34488795, 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.197 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833129+0000", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833130+0000", 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:11.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833135+0000", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8000000000000004e-06 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833156+0000", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0703000000000002e-05 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833167+0000", 2024-09-15T21:01:11.199 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.058e-05 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177702+0000", 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34453553599999998 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178017+0000", 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031424099999999999 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.200 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.267601593999998, 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.345738139, 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.201 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833185+0000", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833186+0000", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9400000000000005e-07 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833189+0000", 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4860000000000002e-06 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.202 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.204 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.204 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833202+0000", 2024-09-15T21:01:11.204 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2116999999999999e-05 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833212+0000", 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0633e-05 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178319+0000", 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34510687200000001 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.205 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178924+0000", 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060443699999999999 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 59/28 ECSubWriteReply(tid=108, last_complete=58'8, committed=1, applied=1))", 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.267402308000001, 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34485581700000001, 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.206 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833385+0000", 2024-09-15T21:01:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5999999999999999e-07 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833388+0000", 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.48e-06 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833399+0000", 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1749999999999999e-05 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.208 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.833406+0000", 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3960000000000003e-06 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178221+0000", 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.34481549900000003 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178241+0000", 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9232000000000002e-05 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.209 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.210 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:11.210 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:11.210 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:11.211 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.116+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:11.211 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.123+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:11.211 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.123+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:11.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:11.212 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:11.207+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:11.222 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.216+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:11.222 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.216+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3 to read 0,1,4,5 2024-09-15T21:01:11.222 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.216+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,4,5 2024-09-15T21:01:11.226 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.220+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:11.226 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:11.220+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:01:11.229 INFO:tasks.rados.rados.0.smithi027.stdout:267: expect (ObjNum 77 snap 9 seq_num 77) 2024-09-15T21:01:11.234 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.227+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,7 to read 0,1,4,5 2024-09-15T21:01:11.234 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.231+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5,7 to read 0,1,4,5 2024-09-15T21:01:11.237 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.231+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:11.237 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.231+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:11.237 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.231+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:11.248 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1165135 2024-09-15T21:01:11.260 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 152880 2024-09-15T21:01:11.301 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 618203 2024-09-15T21:01:11.312 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.307+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:11.317 INFO:tasks.rados.rados.0.smithi027.stdout:259: finishing copy_from to smithi02732947-36 2024-09-15T21:01:11.317 INFO:tasks.rados.rados.0.smithi027.stdout:259: got expected ENOENT (src dne) 2024-09-15T21:01:11.318 INFO:tasks.rados.rados.0.smithi027.stdout:259: finishing copy_from racing read to smithi02732947-36 2024-09-15T21:01:11.318 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 14 v 61 (ObjNum 99 snap 17 seq_num 99) dirty exists 2024-09-15T21:01:11.335 INFO:tasks.rados.rados.0.smithi027.stdout:260: finishing rollback tid 0 to smithi02732947-15 2024-09-15T21:01:11.335 INFO:tasks.rados.rados.0.smithi027.stdout:260: finishing rollback tid 1 to smithi02732947-15 2024-09-15T21:01:11.335 INFO:tasks.rados.rados.0.smithi027.stdout:260: finishing rollback tid 2 to smithi02732947-15 2024-09-15T21:01:11.335 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 15 v 21 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-15T21:01:11.339 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.335+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:11.345 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:11.345 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s5 79/59 ECSubWrite(tid=277, reqid=client.4641.0:656, at_version=79'151, trim_to=0'0, roll_forward_to=78'145))", 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:11.278498+0000", 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.066371052, 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066412493000000003, 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278498+0000", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278498+0000", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278500+0000", 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159999999999999e-06 2024-09-15T21:01:11.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278767+0000", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026687299999999999 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278793+0000", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5701999999999999e-05 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278806+0000", 2024-09-15T21:01:11.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2986999999999999e-05 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s5 79/59 ECSubWrite(tid=278, reqid=client.4641.0:657, at_version=79'152, trim_to=0'0, roll_forward_to=78'145))", 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:11.278838+0000", 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.066030650999999996, 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066148442000000002, 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278838+0000", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278838+0000", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278839+0000", 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5600000000000005e-07 2024-09-15T21:01:11.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278961+0000", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012266299999999999 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278972+0000", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0669000000000001e-05 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278979+0000", 2024-09-15T21:01:11.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4139999999999999e-06 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s5 79/59 ECSubWrite(tid=279, reqid=client.4641.0:658, at_version=79'153, trim_to=0'0, roll_forward_to=78'145))", 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:11.278993+0000", 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.065875780999999994, 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.066058424000000004, 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:11.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278993+0000", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278993+0000", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.278993+0000", 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2024-09-15T21:01:11.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.279083+0000", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9367e-05 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.279091+0000", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6289999999999999e-06 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.279097+0000", 2024-09-15T21:01:11.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:01:11.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:11.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:11.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:11.407+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:11.433 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.427+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:11.436 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.431+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:01:11.436 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:11.431+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,5 to read 0,1,4,5 2024-09-15T21:01:11.438 INFO:tasks.rados.rados.0.smithi027.stdout:263: expect (ObjNum 97 snap 17 seq_num 97) 2024-09-15T21:01:11.438 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:11.480 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:11.512 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:11.513 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:11.508+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:11.529 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:11.529 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:11.529 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:11.529 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:11.529 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:11.536 INFO:tasks.rados.rados.0.smithi027.stdout:266: finishing copy_from racing read to smithi02732947-3 2024-09-15T21:01:11.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:11.615 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:11.612+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:11.715 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:11.711+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:11.721 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:11.721 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.426418099000003, 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.722 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:01:11.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:11.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.425115187999999, 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:01:11.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:01:11.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.425035262999998, 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:01:11.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:01:11.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.218551413999997, 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.740 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:01:11.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:01:11.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.217153969000002, 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.746 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:01:11.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.217114742, 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:01:11.752 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:01:11.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:01:11.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.015264274000003, 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:01:11.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.014893458000003, 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:01:11.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.014723795999998, 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:01:11.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.014315605999997, 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:01:11.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.947035761999999, 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:01:11.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.915822859999999, 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.777 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:01:11.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.914053989999999, 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:01:11.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:166 3.9s0 3:9b48fc7f:::smithi02732947-22:head [copy-from ver 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.814029509000001, 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.71698077900000001, 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904687+0000", 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-15T21:01:11.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904696+0000", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9899999999999992e-06 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904709+0000", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2543e-05 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904717+0000", 2024-09-15T21:01:11.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904853+0000", 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013647299999999999 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904974+0000", 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120728 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115180+0000", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210206536 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116911+0000", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730517 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621588+0000", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.504676919 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621665+0000", 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7520000000000003e-05 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:465 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.977032437999998, 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36382091, 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.790 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 465 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741682+0000", 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741703+0000", 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1155999999999999e-05 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741714+0000", 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1283e-05 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741755+0000", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0698000000000002e-05 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741777+0000", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1696e-05 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.798049+0000", 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056272412000000001 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.798323+0000", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027397100000000002 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800518+0000", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002195125 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.806827+0000", 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006308714 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.105282+0000", 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29845517799999999 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.105503+0000", 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022067699999999999 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:466 3.9s0 3:9b48fc7f:::smithi02732947-22:head [rollback e] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.976905836, 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36587998700000002, 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741810+0000", 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7e-06 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741817+0000", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2099999999999996e-06 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741838+0000", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0659e-05 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741854+0000", 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5930000000000002e-05 2024-09-15T21:01:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802675+0000", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060821433000000001 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802967+0000", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029195399999999998 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804598+0000", 2024-09-15T21:01:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016307870000000001 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809146+0000", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004548191 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107499+0000", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298352532 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107688+0000", 2024-09-15T21:01:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018959099999999999 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:467 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.976833498000001, 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36615330200000001, 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741882+0000", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3e-06 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741888+0000", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9170000000000003e-06 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741910+0000", 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1838999999999999e-05 2024-09-15T21:01:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741928+0000", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8016000000000002e-05 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809749+0000", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067821091 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809897+0000", 2024-09-15T21:01:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148312 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810785+0000", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088809800000000003 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.812435+0000", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016501619999999999 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107930+0000", 2024-09-15T21:01:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295494543 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.108034+0000", 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010402400000000001 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:506 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~540672] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.036836541, 2024-09-15T21:01:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.37081094999999997, 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 506 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681880+0000", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682876+0000", 2024-09-15T21:01:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099578100000000005 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682908+0000", 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2138999999999998e-05 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682923+0000", 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5577e-05 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683044+0000", 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121033 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683056+0000", 2024-09-15T21:01:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1732999999999999e-05 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.746713+0000", 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.063656931999999999 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.747027+0000", 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031414700000000003 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.749307+0000", 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022794339999999999 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.841341+0000", 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.092034070999999995 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052501+0000", 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21116006500000001 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052689+0000", 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000187708 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:508 3.9s0 3:9b48fc7f:::smithi02732947-22:head [setxattr _header (57) in=7b] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.027775975000001, 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.459624861, 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 508 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690939+0000", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690947+0000", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690958+0000", 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.128e-05 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690967+0000", 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691124+0000", 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015739899999999999 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691145+0000", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942749+0000", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25160399300000003 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942917+0000", 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167522 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.943821+0000", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090415099999999996 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.945525+0000", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017036480000000001 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150355+0000", 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20482993199999999 2024-09-15T21:01:11.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150563+0000", 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020850400000000001 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:624 3.9s0 3:9b0039f5:::smithi02732947-45:head [create,setxattr [OXS[Rf]kSmSVZkcqrPoTRUsq[o]epTblNRaQggt (1858) in=40b,setxattr QrooaVmb\\N]q[ho]kgWWcr\\eocaQRtZ_`]Qsipqt (174) in=40b,setxattr Ydefbt^YQ\\dpmjectRYX^qRe[eQdQqbZhllc (770) in=36b,setxattr TV\\gUfWjiSV[PabStYoPOXcXUVsSaXdhhRnnhpfs (454) in=40b,setxattr m_O\\\\icSfcPOnoNRZ]O`X]X`ZaVOXegXNVmPglmp (734) in=40b,setxattr mf`_l`a[Y`agdjhtOpoWmnWT^dS[ScSdZWamrcjp (699) in=40b,setxattr hPm]lVO[\\[iefNWiNnaWNQr (770) in=23b,setxattr gkna_]qPX[S`[_g_ifYo]hUlgScp_jnl[^WpdheP (1614) in=40b,setxattr YSndfaQqlVQQXmZiNjPrZZa`glYedsbkkcroSttS (1333) in=40b,setxattr VVliPgssabe (770) in=11b,setxattr kdelpR\\kNn^\\r\\aeabqa\\jcQSaOXgNlVTroTaodb (1610) in=40b,setxattr a[YZZ[_nQae][Ujatlhnen_QorP_PROiQkaafPl^ (1754) in=40b,setxattr Y\\iYcXa`paqfZggTnSmaVZ[Yt`te\\rdepg^foUUX (12280) in=40b,setxattr WPicPk_V\\Z]`USR]\\Xq (770) in=19b,setxattr gqtnZjUUjhkmnQqhohYVhkliPOppPSXTshf_sVbq (877) in=40b,setxattr Nadgio[acb^P_fdps]bigQrctbcktsZdTe]Z_hhb (8628) in=40b,setxattr oUUbUt^qtWedq]`NoRbZf[jPhdNrRbjTU`m^jbpe (666) in=40b,setxattr UUhYqmbd_S[all[PkPg\\odhP\\gkshjetOj][^OWq (16140) in=40b,setxattr WSTftae_cS`q^sZiZQskrTnpRX^Tkmid^ZbYOeTo (1997) in=40b,setxattr bdcqYYhcTQq^O[TjlUb[oTXa_hoaQStVglem^VqX (981) in=40b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.0818542730000003, 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50056462599999996, 2024-09-15T21:01:11.819 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 624 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636862+0000", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637027+0000", 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016484200000000001 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637047+0000", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0182000000000001e-05 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637059+0000", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1892999999999999e-05 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745449+0000", 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10838987999999999 2024-09-15T21:01:11.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745728+0000", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027862699999999999 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.747309+0000", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001581004 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.748624+0000", 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013150099999999999 2024-09-15T21:01:11.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137271+0000", 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38864758999999999 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137424+0000", 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015299899999999999 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:11.824 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:11.825 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:11.826 INFO:tasks.rados.rados.0.smithi027.stdout:266: finishing copy_from to smithi02732947-3 2024-09-15T21:01:11.826 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 3 v 128 (ObjNum 105 snap 17 seq_num 105) dirty exists 2024-09-15T21:01:11.827 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 1 to smithi02732947-50 2024-09-15T21:01:11.827 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 2 to smithi02732947-50 2024-09-15T21:01:11.827 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 3 to smithi02732947-50 2024-09-15T21:01:11.827 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 4 to smithi02732947-50 2024-09-15T21:01:11.827 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 5 to smithi02732947-50 2024-09-15T21:01:11.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:11.828 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:11.820+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:11.851 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 1 to smithi02732947-27 2024-09-15T21:01:11.852 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 2 to smithi02732947-27 2024-09-15T21:01:11.854 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:11.847+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:11.927 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:11.924+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:11.953 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:11.947+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:01:11.954 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:11.947+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,7 to read 0,1,4,5 2024-09-15T21:01:11.954 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:11.947+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: finishing write tid 6 to smithi02732947-50 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: oid 50 updating version 0 to 152 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: oid 50 version 152 is already newer than 151 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: oid 50 updating version 152 to 153 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: oid 50 updating version 153 to 154 2024-09-15T21:01:11.955 INFO:tasks.rados.rados.0.smithi027.stdout:262: oid 50 updating version 154 to 155 2024-09-15T21:01:11.956 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 50 v 155 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-15T21:01:11.956 INFO:tasks.rados.rados.0.smithi027.stdout:262: left oid 50 (ObjNum 116 snap 22 seq_num 116) 2024-09-15T21:01:11.999 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:11.999 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:11.999 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:12.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:12.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:12.058 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:12.055+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:12.128 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:12.123+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:12.136 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:12.136 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:12.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:12.174 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:12.174 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:12.192 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:12.192 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:12.211 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:12.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:12.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.223+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:12.249 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:12.249 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:12.270 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:12.320 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:12.320 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:12.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:12.320 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 33/28 ECSubRead(tid=24, to_read={3:29dd4e88:::smithi02732947-38:head=0,4096,0}, subchunks={3:29dd4e88:::smithi02732947-38:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.82445053, 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20085125200000001, 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:12.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492667+0000", 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492668+0000", 2024-09-15T21:01:12.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.17e-06 2024-09-15T21:01:12.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492688+0000", 2024-09-15T21:01:12.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9755999999999999e-05 2024-09-15T21:01:12.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.492721+0000", 2024-09-15T21:01:12.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2561000000000002e-05 2024-09-15T21:01:12.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.592869+0000", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100148689 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693164+0000", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10029479600000001 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.693518+0000", 2024-09-15T21:01:12.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035428 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.201572399, 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20249605200000001, 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115546+0000", 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5900000000000005e-07 2024-09-15T21:01:12.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115662+0000", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011552699999999999 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115676+0000", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4547e-05 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215777+0000", 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100100516 2024-09-15T21:01:12.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215874+0000", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7094000000000002e-05 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215886+0000", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759999999999999e-05 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.317999+0000", 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102113494 2024-09-15T21:01:12.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318041+0000", 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2154999999999999e-05 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.943964348999998, 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:01:12.332 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:01:12.336 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.843484265000001, 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20147183599999999, 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473634+0000", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473665+0000", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473707+0000", 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1860999999999998e-05 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473724+0000", 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7206000000000001e-05 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673974+0000", 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20025005600000001 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673991+0000", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6368999999999999e-05 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675017+0000", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001026267 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675105+0000", 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8195000000000005e-05 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=41, reqid=client.4641.0:360, at_version=48'16, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.764216511000001, 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20315024000000001, 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:12.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552903+0000", 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.561e-06 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553100+0000", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019769300000000001 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553123+0000", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3008999999999999e-05 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653267+0000", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014326899999999 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753534+0000", 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100267523 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753550+0000", 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5824999999999998e-05 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755996+0000", 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024464930000000001 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756051+0000", 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4867000000000002e-05 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:12.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.553568457000001, 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30705592100000001, 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:12.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763551+0000", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763784+0000", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023310900000000001 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763818+0000", 2024-09-15T21:01:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3855e-05 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763835+0000", 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6393e-05 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964212+0000", 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200376947 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964244+0000", 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2035000000000001e-05 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070501+0000", 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10625775599999999 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070605+0000", 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103706 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=5, reqid=client.4641.0:430, at_version=59'75, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.548294127999998, 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.402708554, 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768826+0000", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000002e-06 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769062+0000", 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581599999999999 2024-09-15T21:01:12.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769156+0000", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4553000000000001e-05 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769216+0000", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9932000000000001e-05 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966630+0000", 2024-09-15T21:01:12.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19741336600000001 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966643+0000", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3609e-05 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171495+0000", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20485210600000001 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171532+0000", 2024-09-15T21:01:12.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6711999999999997e-05 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.445324082999999, 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299792744, 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871820+0000", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5873000000000001e-05 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871847+0000", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6792e-05 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871873+0000", 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6075999999999999e-05 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068219+0000", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19634641799999999 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068236+0000", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6912999999999999e-05 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171560+0000", 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10332366 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171586+0000", 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6438999999999999e-05 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 21.445068938999999, 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29954117000000002, 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872055+0000", 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8370000000000002e-06 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872067+0000", 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493e-05 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872079+0000", 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1456e-05 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171223+0000", 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299144415 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171590+0000", 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036652199999999998 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=57, reqid=client.4641.0:465, at_version=60'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.516313706999998, 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30393555799999999, 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-07 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800860+0000", 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5575000000000002e-05 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800883+0000", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3116e-05 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800903+0000", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9426e-05 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001512+0000", 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20060972299999999 2024-09-15T21:01:12.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001540+0000", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7674999999999998e-05 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104682+0000", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103142055 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104739+0000", 2024-09-15T21:01:12.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7481000000000001e-05 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=58, reqid=client.4641.0:466, at_version=60'38, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.512450583, 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30238109299999999, 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804669+0000", 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7969999999999999e-06 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804721+0000", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1820999999999997e-05 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804762+0000", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1115999999999997e-05 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804786+0000", 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4352e-05 2024-09-15T21:01:12.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103533+0000", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29874738099999998 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103546+0000", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2768e-05 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.106961+0000", 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415094 2024-09-15T21:01:12.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107048+0000", 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6763999999999998e-05 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=59, reqid=client.4641.0:467, at_version=60'39, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.506297397000001, 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29628899199999997, 2024-09-15T21:01:12.370 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810822+0000", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0430000000000002e-06 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810872+0000", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9962e-05 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810913+0000", 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0642000000000002e-05 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810940+0000", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6662e-05 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104296+0000", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29335650600000002 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104312+0000", 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5559000000000001e-05 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107057+0000", 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027455829999999998 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107109+0000", 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2034999999999999e-05 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:12.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.417146628000001, 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20751788099999999, 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:12.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899973+0000", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900092+0000", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118466 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900123+0000", 2024-09-15T21:01:12.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1149000000000003e-05 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900146+0000", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779000000000001e-05 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900331+0000", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000184594 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900362+0000", 2024-09-15T21:01:12.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0559000000000003e-05 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107409+0000", 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20704719299999999 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107489+0000", 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0061000000000005e-05 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=16, reqid=client.4641.0:462, at_version=60'83, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.416921778999999, 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20735974700000001, 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900197+0000", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900419+0000", 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022236900000000001 2024-09-15T21:01:12.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900447+0000", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7373e-05 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900472+0000", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981999999999999e-05 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002446+0000", 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101974907 2024-09-15T21:01:12.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002464+0000", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045999999999999e-05 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107496+0000", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105032797 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107556+0000", 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259000000000003e-05 2024-09-15T21:01:12.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=17, reqid=client.4641.0:463, at_version=60'84, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.416599476000002, 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20715383400000001, 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900520+0000", 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900569+0000", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9608999999999999e-05 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900593+0000", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4034999999999999e-05 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900613+0000", 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103732+0000", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.203119718 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103749+0000", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107564+0000", 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038144160000000002 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107672+0000", 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108294 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 60/28 ECSubWrite(tid=183, reqid=client.4641.0:448, at_version=60'44, trim_to=0'0, roll_forward_to=56'42))", 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.408519934000001, 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20201134200000001, 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.387 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:12.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910197+0000", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015987340000000001 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910229+0000", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2623000000000003e-05 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910247+0000", 2024-09-15T21:01:12.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7075000000000002e-05 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104183+0000", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19393659899999999 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104208+0000", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499e-05 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110513+0000", 2024-09-15T21:01:12.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063052070000000002 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110609+0000", 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6191000000000001e-05 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs7 68/59 ECSubWrite(tid=182, reqid=client.4641.0:570, at_version=0'0, trim_to=0'0, roll_forward_to=68'70))", 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.014213505000001, 2024-09-15T21:01:12.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200853328, 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302904+0000", 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302906+0000", 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.95e-06 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302929+0000", 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2852000000000001e-05 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.302959+0000", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9836999999999999e-05 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403039+0000", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10008046499999999 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403103+0000", 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4040000000000003e-05 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.403126+0000", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2694999999999999e-05 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503703+0000", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10057682699999999 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:00.503757+0000", 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4661999999999999e-05 2024-09-15T21:01:12.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.1s5 73/71 ECSubWrite(tid=111, reqid=osd.0.0:110, at_version=0'0, trim_to=0'0, roll_forward_to=73'34))", 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9459908969999997, 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202398409, 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371129+0000", 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6000000000000001e-06 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371161+0000", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1343000000000001e-05 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371203+0000", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1943000000000001e-05 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371228+0000", 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5338999999999999e-05 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371405+0000", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177472 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371436+0000", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1016000000000003e-05 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.573382+0000", 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20194593599999999 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.573525+0000", 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014276 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 73/72 ECSubWrite(tid=91, reqid=osd.8.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=73'55))", 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 6.9416975089999999, 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29881405799999999, 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.400 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375421+0000", 2024-09-15T21:01:12.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375434+0000", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2388000000000001e-05 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375447+0000", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2979000000000001e-05 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375456+0000", 2024-09-15T21:01:12.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8870000000000004e-06 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572431+0000", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19697500200000001 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572450+0000", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8260000000000001e-05 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674174+0000", 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10172400700000001 2024-09-15T21:01:12.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674234+0000", 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0445000000000001e-05 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 79/57 ECSubRead(tid=77, to_read={3:26d7dfe1:::smithi02732947-26:head=0,499712,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:12.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.078881035, 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20146718899999999, 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:12.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238238+0000", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0219999999999999e-06 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238250+0000", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2455e-05 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238277+0000", 2024-09-15T21:01:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7385e-05 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.338396+0000", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100118153 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.438594+0000", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100198367 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.439704+0000", 2024-09-15T21:01:12.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001109807 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:12.408 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:12.409 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:12.409 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:12.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:12.410 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:12.403+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:12.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:12.416 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:12.449 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:12.449 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:12.479 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:12.479 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:12.505 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:12.505 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:12.510 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:12.508+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:12.532 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:12.566 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 3 to smithi02732947-27 2024-09-15T21:01:12.567 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 4 to smithi02732947-27 2024-09-15T21:01:12.583 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:12.583 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:12.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:12.627 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:01:12.642 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:12.639+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:12.667 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 5 to smithi02732947-27 2024-09-15T21:01:12.670 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.663+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:12.670 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.667+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:12.713 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:12.707+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:12.720 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:12.721 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:12.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:12.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.763+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: finishing write tid 6 to smithi02732947-27 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: oid 27 updating version 0 to 158 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: oid 27 updating version 158 to 161 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: oid 27 version 161 is already newer than 160 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: oid 27 version 161 is already newer than 157 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: oid 27 version 161 is already newer than 159 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 27 v 161 (ObjNum 117 snap 22 seq_num 117) dirty exists 2024-09-15T21:01:12.770 INFO:tasks.rados.rados.0.smithi027.stdout:264: left oid 27 (ObjNum 117 snap 22 seq_num 117) 2024-09-15T21:01:12.814 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:12.807+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:12.869 INFO:tasks.rados.rados.0.smithi027.stdout:259: done (8 left) 2024-09-15T21:01:12.869 INFO:tasks.rados.rados.0.smithi027.stdout:260: done (7 left) 2024-09-15T21:01:12.869 INFO:tasks.rados.rados.0.smithi027.stdout:261: done (6 left) 2024-09-15T21:01:12.869 INFO:tasks.rados.rados.0.smithi027.stdout:262: done (5 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:263: done (4 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:264: done (3 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:266: done (2 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:267: done (1 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:269: done (0 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:270: copy_from oid 32 from oid 23 current snap is 22 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:271: read oid 7 snap -1 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:271: expect deleted 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:272: copy_from oid 2 from oid 41 current snap is 22 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:273: rmattr oid 46 current snap is 22 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:273: done (3 left) 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:274: read oid 46 snap -1 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:274: expect deleted 2024-09-15T21:01:12.870 INFO:tasks.rados.rados.0.smithi027.stdout:275: copy_from oid 17 from oid 49 current snap is 22 2024-09-15T21:01:12.871 INFO:tasks.rados.rados.0.smithi027.stdout:276: copy_from oid 6 from oid 22 current snap is 22 2024-09-15T21:01:12.871 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 9 current snap is 22 2024-09-15T21:01:12.871 INFO:tasks.rados.rados.0.smithi027.stdout:277: seq_num 119 ranges {2179072=458752,2637824=507904,3145728=507904,3653632=458752,4112384=81920} 2024-09-15T21:01:12.875 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:12.871+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:12.881 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.875+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:12.893 INFO:tasks.rados.rados.0.smithi027.stdout:277: writing smithi02732947-9 from 2179072 to 2637824 tid 1 2024-09-15T21:01:12.897 INFO:tasks.rados.rados.0.smithi027.stdout:277: writing smithi02732947-9 from 2637824 to 3145728 tid 2 2024-09-15T21:01:12.902 INFO:tasks.rados.rados.0.smithi027.stdout:277: writing smithi02732947-9 from 3145728 to 3653632 tid 3 2024-09-15T21:01:12.906 INFO:tasks.rados.rados.0.smithi027.stdout:277: writing smithi02732947-9 from 3653632 to 4112384 tid 4 2024-09-15T21:01:12.907 INFO:tasks.rados.rados.0.smithi027.stdout:277: writing smithi02732947-9 from 4112384 to 4194304 tid 5 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:278: read oid 43 snap -1 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:278: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:279: read oid 42 snap -1 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:279: expect deleted 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:280: rollback oid 30 current snap is 22 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 30 to 20 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:271: done (9 left) 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:274: done (8 left) 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:281: rollback oid 25 current snap is 22 2024-09-15T21:01:12.908 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 25 to 19 2024-09-15T21:01:12.909 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 18 current snap is 22 2024-09-15T21:01:12.909 INFO:tasks.rados.rados.0.smithi027.stdout:282: seq_num 120 ranges {4308992=475136,4784128=655360,5439488=622592,6062080=278528} 2024-09-15T21:01:12.909 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:12.909 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:12.910 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:12.904+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:12.910 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:12.904+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:12.915 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:12.911+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:12.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:12.948 INFO:tasks.rados.rados.0.smithi027.stdout:282: writing smithi02732947-18 from 4308992 to 4784128 tid 1 2024-09-15T21:01:12.953 INFO:tasks.rados.rados.0.smithi027.stdout:282: writing smithi02732947-18 from 4784128 to 5439488 tid 2 2024-09-15T21:01:12.958 INFO:tasks.rados.rados.0.smithi027.stdout:282: writing smithi02732947-18 from 5439488 to 6062080 tid 3 2024-09-15T21:01:12.960 INFO:tasks.rados.rados.0.smithi027.stdout:282: writing smithi02732947-18 from 6062080 to 6340608 tid 4 2024-09-15T21:01:12.960 INFO:tasks.rados.rados.0.smithi027.stdout:283: read oid 5 snap -1 2024-09-15T21:01:12.960 INFO:tasks.rados.rados.0.smithi027.stdout:283: expect deleted 2024-09-15T21:01:12.961 INFO:tasks.rados.rados.0.smithi027.stdout:284: snap_create 2024-09-15T21:01:12.970 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.963+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:12.975 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:12.971+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:12.975 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:12.971+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,4,5 2024-09-15T21:01:12.977 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:12.971+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:13.010 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.004+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:13.011 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.008+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3 to read 0,1,4,5 2024-09-15T21:01:13.011 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.008+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:01:13.017 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:13.012+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:13.036 INFO:tasks.rados.rados.0.smithi027.stdout:272: finishing copy_from racing read to smithi02732947-2 2024-09-15T21:01:13.067 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:13.063+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:13.068 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:13.063+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:13.068 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:13.063+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:13.100 INFO:tasks.rados.rados.0.smithi027.stdout:276: finishing copy_from racing read to smithi02732947-6 2024-09-15T21:01:13.102 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.102 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.114 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.108+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:13.114 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.108+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:13.115 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.112+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:01:13.115 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.112+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:01:13.116 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:13.111+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:13.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:13.158 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.152+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:13.165 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.160+0000 7f885ca8f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6 to read 0,1,4,5 2024-09-15T21:01:13.170 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.164+0000 7f8858a87640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,6 to read 0,1,4,5 2024-09-15T21:01:13.205 INFO:tasks.rados.rados.0.smithi027.stdout:270: finishing copy_from racing read to smithi02732947-32 2024-09-15T21:01:13.214 INFO:tasks.rados.rados.0.smithi027.stdout:278: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T21:01:13.214 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:13.217 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:13.211+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:13.232 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:13.250 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:13.284 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.284 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:13.319 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:13.315+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:13.372 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 1 to smithi02732947-18 2024-09-15T21:01:13.381 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 1 to smithi02732947-9 2024-09-15T21:01:13.419 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:13.416+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:13.437 INFO:tasks.rados.rados.0.smithi027.stdout:272: finishing copy_from to smithi02732947-2 2024-09-15T21:01:13.437 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 2 v 76 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-15T21:01:13.464 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 2 to smithi02732947-18 2024-09-15T21:01:13.466 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.466 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.481 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 2 to smithi02732947-9 2024-09-15T21:01:13.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:13.520 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:13.515+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:13.539 INFO:tasks.rados.rados.0.smithi027.stdout:276: finishing copy_from to smithi02732947-6 2024-09-15T21:01:13.539 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 48 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-15T21:01:13.564 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 3 to smithi02732947-18 2024-09-15T21:01:13.582 INFO:tasks.rados.rados.0.smithi027.stdout:281: finishing rollback tid 0 to smithi02732947-25 2024-09-15T21:01:13.582 INFO:tasks.rados.rados.0.smithi027.stdout:281: finishing rollback tid 1 to smithi02732947-25 2024-09-15T21:01:13.583 INFO:tasks.rados.rados.0.smithi027.stdout:281: finishing rollback tid 2 to smithi02732947-25 2024-09-15T21:01:13.583 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 166 (ObjNum 107 snap 18 seq_num 107) dirty exists 2024-09-15T21:01:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:272: done (11 left) 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:276: done (10 left) 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:278: done (9 left) 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:279: done (8 left) 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:281: done (7 left) 2024-09-15T21:01:13.618 INFO:tasks.rados.rados.0.smithi027.stdout:283: done (6 left) 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:284: done (5 left) 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:285: rollback oid 21 current snap is 23 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 21 to 22 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:286: rmattr oid 29 current snap is 23 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 33 current snap is 23 2024-09-15T21:01:13.619 INFO:tasks.rados.rados.0.smithi027.stdout:287: seq_num 121 ranges {2441216=491520,2932736=802816,3735552=720896,4456448=32768} 2024-09-15T21:01:13.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:13.615+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:13.624 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.624 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.638 INFO:tasks.rados.rados.0.smithi027.stdout:287: writing smithi02732947-33 from 2441216 to 2932736 tid 1 2024-09-15T21:01:13.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:13.643 INFO:tasks.rados.rados.0.smithi027.stdout:287: writing smithi02732947-33 from 2932736 to 3735552 tid 2 2024-09-15T21:01:13.648 INFO:tasks.rados.rados.0.smithi027.stdout:287: writing smithi02732947-33 from 3735552 to 4456448 tid 3 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:287: writing smithi02732947-33 from 4456448 to 4489216 tid 4 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:288: copy_from oid 36 from oid 25 current snap is 23 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:280: finishing rollback tid 0 to smithi02732947-30 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:280: finishing rollback tid 1 to smithi02732947-30 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:280: finishing rollback tid 2 to smithi02732947-30 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 140 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:280: done (8 left) 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:289: rmattr oid 38 current snap is 23 2024-09-15T21:01:13.649 INFO:tasks.rados.rados.0.smithi027.stdout:289: done (8 left) 2024-09-15T21:01:13.652 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 41 current snap is 23 2024-09-15T21:01:13.652 INFO:tasks.rados.rados.0.smithi027.stdout:290: seq_num 122 ranges {3260416=507904,3768320=425984,4194304=442368,4636672=540672,5177344=147456} 2024-09-15T21:01:13.666 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.660+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:13.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.660+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6 to read 0,1,4,5 2024-09-15T21:01:13.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.660+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:01:13.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.660+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6,7 to read 0,1,4,5 2024-09-15T21:01:13.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:13.660+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,4,5 2024-09-15T21:01:13.673 INFO:tasks.rados.rados.0.smithi027.stdout:290: writing smithi02732947-41 from 3260416 to 3768320 tid 1 2024-09-15T21:01:13.676 INFO:tasks.rados.rados.0.smithi027.stdout:290: writing smithi02732947-41 from 3768320 to 4194304 tid 2 2024-09-15T21:01:13.680 INFO:tasks.rados.rados.0.smithi027.stdout:290: writing smithi02732947-41 from 4194304 to 4636672 tid 3 2024-09-15T21:01:13.683 INFO:tasks.rados.rados.0.smithi027.stdout:290: writing smithi02732947-41 from 4636672 to 5177344 tid 4 2024-09-15T21:01:13.685 INFO:tasks.rados.rados.0.smithi027.stdout:290: writing smithi02732947-41 from 5177344 to 5324800 tid 5 2024-09-15T21:01:13.685 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 34 current snap is 23 2024-09-15T21:01:13.685 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 4 to smithi02732947-18 2024-09-15T21:01:13.685 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 5 to smithi02732947-18 2024-09-15T21:01:13.685 INFO:tasks.rados.rados.0.smithi027.stdout:291: seq_num 123 ranges {2342912=524288,2867200=573440,3440640=688128,4128768=294912} 2024-09-15T21:01:13.702 INFO:tasks.rados.rados.0.smithi027.stdout:291: writing smithi02732947-34 from 2342912 to 2867200 tid 1 2024-09-15T21:01:13.706 INFO:tasks.rados.rados.0.smithi027.stdout:291: writing smithi02732947-34 from 2867200 to 3440640 tid 2 2024-09-15T21:01:13.710 INFO:tasks.rados.rados.0.smithi027.stdout:291: writing smithi02732947-34 from 3440640 to 4128768 tid 3 2024-09-15T21:01:13.712 INFO:tasks.rados.rados.0.smithi027.stdout:291: writing smithi02732947-34 from 4128768 to 4423680 tid 4 2024-09-15T21:01:13.712 INFO:tasks.rados.rados.0.smithi027.stdout:292: copy_from oid 1 from oid 6 current snap is 23 2024-09-15T21:01:13.712 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 30 current snap is 23 2024-09-15T21:01:13.712 INFO:tasks.rados.rados.0.smithi027.stdout:293: seq_num 124 ranges {491520=540672,1032192=720896,1753088=786432,2539520=49152} 2024-09-15T21:01:13.718 INFO:tasks.rados.rados.0.smithi027.stdout:293: writing smithi02732947-30 from 491520 to 1032192 tid 1 2024-09-15T21:01:13.721 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:13.716+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:13.722 INFO:tasks.rados.rados.0.smithi027.stdout:293: writing smithi02732947-30 from 1032192 to 1753088 tid 2 2024-09-15T21:01:13.727 INFO:tasks.rados.rados.0.smithi027.stdout:293: writing smithi02732947-30 from 1753088 to 2539520 tid 3 2024-09-15T21:01:13.727 INFO:tasks.rados.rados.0.smithi027.stdout:293: writing smithi02732947-30 from 2539520 to 2588672 tid 4 2024-09-15T21:01:13.727 INFO:tasks.rados.rados.0.smithi027.stdout:294: snap_remove snap 19 2024-09-15T21:01:13.776 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.777 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:13.822 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:13.819+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:13.891 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:13.887+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:13.891 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:13.887+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:01:13.892 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:13.887+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:01:13.923 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:13.919+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:13.966 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:13.966 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:13.968 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:13.963+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,6,7 to read 0,1,4,5 2024-09-15T21:01:13.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:14.027 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:14.023+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:14.075 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:14.071+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:14.075 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:14.071+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:14.075 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:14.071+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4 to read 0,1,4,5 2024-09-15T21:01:14.075 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:14.071+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4,7 to read 0,1,4,5 2024-09-15T21:01:14.113 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:14.113 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:14.124 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.119+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:14.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:14.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.219+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:14.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:14.287+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:14.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:14.287+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:01:14.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:14.287+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:01:14.298 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:14.298 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:14.313 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:14.326 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.323+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:14.390 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:14.383+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:14.390 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:14.383+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:14.428 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.423+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:14.441 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:14.442 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:14.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:14.473 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:14.467+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:14.473 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:14.467+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:01:14.490 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:14.487+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:01:14.491 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:14.487+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:01:14.491 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:14.487+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,6 to read 0,1,4,5 2024-09-15T21:01:14.529 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:14.527+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:14.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:14.573 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:14.567+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:01:14.574 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:14.571+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,6,7 to read 0,1,4,5 2024-09-15T21:01:14.578 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.575+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:14.578 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.575+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:01:14.584 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:14.585 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:14.592 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.587+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,4,5 2024-09-15T21:01:14.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:294: done (12 left) 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:295: copy_from oid 37 from oid 16 current snap is 23 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:275: finishing copy_from racing read to smithi02732947-17 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:270: finishing copy_from to smithi02732947-32 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 32 v 110 (ObjNum 96 snap 17 seq_num 96) dirty exists 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 3 to smithi02732947-9 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 1 to smithi02732947-33 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 2 to smithi02732947-33 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 29 v 40 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T21:01:14.619 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 4 to smithi02732947-9 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 5 to smithi02732947-9 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:292: finishing copy_from racing read to smithi02732947-1 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:285: finishing rollback tid 0 to smithi02732947-21 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:285: finishing rollback tid 1 to smithi02732947-21 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:285: finishing rollback tid 2 to smithi02732947-21 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 21 v 74 (ObjNum 76 snap 8 seq_num 76) dirty exists 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 6 to smithi02732947-9 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: finishing write tid 6 to smithi02732947-18 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: oid 18 updating version 0 to 85 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: oid 18 updating version 85 to 86 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: oid 18 updating version 86 to 89 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: oid 18 version 89 is already newer than 87 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:282: oid 18 version 89 is already newer than 88 2024-09-15T21:01:14.620 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 89 (ObjNum 120 snap 22 seq_num 120) dirty exists 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:282: left oid 18 (ObjNum 120 snap 22 seq_num 120) 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 3 to smithi02732947-33 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 4 to smithi02732947-33 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 1 to smithi02732947-34 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 2 to smithi02732947-34 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 3 to smithi02732947-34 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 1 to smithi02732947-41 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:270: done (12 left) 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:282: done (11 left) 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:285: done (10 left) 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:286: done (9 left) 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:296: copy_from oid 5 from oid 15 current snap is 23 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 4 to smithi02732947-34 2024-09-15T21:01:14.621 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 5 to smithi02732947-34 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:292: finishing copy_from to smithi02732947-1 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 1 v 48 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:275: finishing copy_from to smithi02732947-17 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 17 v 61 (ObjNum 92 snap 14 seq_num 92) dirty exists 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 2 to smithi02732947-41 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 5 to smithi02732947-33 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 1 to smithi02732947-30 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: finishing write tid 7 to smithi02732947-9 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 updating version 0 to 65 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 version 65 is already newer than 63 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 version 65 is already newer than 62 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 updating version 65 to 67 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 version 67 is already newer than 66 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:277: oid 9 version 67 is already newer than 64 2024-09-15T21:01:14.622 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 67 (ObjNum 119 snap 22 seq_num 119) dirty exists 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:277: left oid 9 (ObjNum 119 snap 22 seq_num 119) 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: finishing write tid 6 to smithi02732947-34 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: oid 34 updating version 0 to 12 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: oid 34 updating version 12 to 13 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: oid 34 version 13 is already newer than 9 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: oid 34 version 13 is already newer than 10 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: oid 34 version 13 is already newer than 11 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 34 v 13 (ObjNum 123 snap 23 seq_num 123) dirty exists 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:291: left oid 34 (ObjNum 123 snap 23 seq_num 123) 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 2 to smithi02732947-30 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 3 to smithi02732947-30 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 4 to smithi02732947-30 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 5 to smithi02732947-30 2024-09-15T21:01:14.623 INFO:tasks.rados.rados.0.smithi027.stdout:287: finishing write tid 6 to smithi02732947-33 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: oid 33 updating version 0 to 64 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: oid 33 version 64 is already newer than 63 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: oid 33 updating version 64 to 67 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: oid 33 version 67 is already newer than 65 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: oid 33 version 67 is already newer than 66 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 33 v 67 (ObjNum 121 snap 23 seq_num 121) dirty exists 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: left oid 33 (ObjNum 121 snap 23 seq_num 121) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:275: done (9 left) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:277: done (8 left) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:287: done (7 left) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:291: done (6 left) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:292: done (5 left) 2024-09-15T21:01:14.624 INFO:tasks.rados.rados.0.smithi027.stdout:297: snap_create 2024-09-15T21:01:14.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:14.663+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:14.673 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:14.667+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,7 to read 0,1,4,5 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: finishing write tid 6 to smithi02732947-30 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: oid 30 updating version 0 to 145 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: oid 30 updating version 145 to 146 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: oid 30 version 146 is already newer than 143 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: oid 30 version 146 is already newer than 144 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: oid 30 version 146 is already newer than 142 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 146 (ObjNum 124 snap 23 seq_num 124) dirty exists 2024-09-15T21:01:14.695 INFO:tasks.rados.rados.0.smithi027.stdout:293: left oid 30 (ObjNum 124 snap 23 seq_num 124) 2024-09-15T21:01:14.729 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:14.723+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:14.737 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.731+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:14.742 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:14.742 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:14.747 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:14.747 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:14.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:14.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:14.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:14.783 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:14.800 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:14.800 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:14.820 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:14.821 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:14.830 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:14.823+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:14.838 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.831+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:01:14.838 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.831+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:01:14.838 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.835+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,7 to read 0,1,4,5 2024-09-15T21:01:14.843 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:14.835+0000 7f61cfb42640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 15 after trimming was started 2024-09-15T21:01:14.877 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 3 to smithi02732947-41 2024-09-15T21:01:14.897 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:14.897 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.299458246, 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:14.902 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 45.811505857, 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:14.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:01:14.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:01:14.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 44.108492226999999, 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:01:14.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:01:14.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 43.522368929000002, 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104322594, 2024-09-15T21:01:14.911 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372641+0000", 2024-09-15T21:01:14.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372642+0000", 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7069999999999999e-06 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372856+0000", 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000213255 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.372963+0000", 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000107661 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473110+0000", 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100146978 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473309+0000", 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019897599999999999 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.473331+0000", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1427999999999999e-05 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476903+0000", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035721609999999999 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.476963+0000", 2024-09-15T21:01:14.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0427999999999998e-05 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 48/28 ECSubRead(tid=132, to_read={3:a5da2594:::smithi02732947-25:head=0,315392,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 34.431974742999998, 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20088298299999999, 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463043+0000", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463058+0000", 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4732999999999999e-05 2024-09-15T21:01:14.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.563142+0000", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008391699999999 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663346+0000", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100204423 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663918+0000", 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057127899999999995 2024-09-15T21:01:14.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.023951187000002, 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30162102800000001, 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871059+0000", 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:01:14.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871100+0000", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0321999999999998e-05 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871128+0000", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8208000000000002e-05 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871152+0000", 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4246000000000001e-05 2024-09-15T21:01:14.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971464+0000", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312292 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971482+0000", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7176e-05 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172613+0000", 2024-09-15T21:01:14.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20113102099999999 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172679+0000", 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6762999999999998e-05 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.023646328000002, 2024-09-15T21:01:14.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20089554500000001, 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:14.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871364+0000", 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871378+0000", 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3706000000000001e-05 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871408+0000", 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0574999999999998e-05 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971547+0000", 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013905400000001 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072019+0000", 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100471385 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072259+0000", 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023990200000000001 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.996012816, 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20140428699999999, 2024-09-15T21:01:14.928 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898998+0000", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4980000000000001e-06 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899107+0000", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010874699999999999 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899130+0000", 2024-09-15T21:01:14.930 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.268e-05 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999226+0000", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096764 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999366+0000", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139231 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999377+0000", 2024-09-15T21:01:14.931 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1097000000000001e-05 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100334+0000", 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100957065 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100401+0000", 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7205000000000005e-05 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=180, reqid=client.4641.0:451, at_version=60'74, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.994234256999999, 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.200850169, 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900776+0000", 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.934 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900867+0000", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1712000000000006e-05 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900880+0000", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.218e-05 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001054+0000", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10017451600000001 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.935 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001292+0000", 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023806500000000001 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001309+0000", 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101581+0000", 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027267099999999 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.936 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101625+0000", 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3965999999999999e-05 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=182, reqid=client.4641.0:453, at_version=60'76, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.893778545, 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10391302099999999, 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001231+0000", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.938 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001232+0000", 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7599999999999997e-07 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001254+0000", 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2178e-05 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001269+0000", 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5576000000000001e-05 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001282+0000", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2642e-05 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101230+0000", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099947724000000002 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101250+0000", 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105105+0000", 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038547550000000001 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105144+0000", 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9075000000000001e-05 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.941 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 60/28 ECSubWrite(tid=183, reqid=client.4641.0:448, at_version=60'44, trim_to=0'0, roll_forward_to=56'42))", 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 22.893708207, 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104610525, 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001301+0000", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001302+0000", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9500000000000002e-07 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001809+0000", 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050734700000000001 2024-09-15T21:01:14.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001828+0000", 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8417999999999999e-05 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101908+0000", 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008057200000001 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101977+0000", 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8571999999999993e-05 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101983+0000", 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1990000000000003e-06 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105842+0000", 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038594150000000002 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.105912+0000", 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9406999999999995e-05 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 68/53 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=68'11, trim_to=0'0, roll_forward_to=67'9))", 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.147234012, 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20168053699999999, 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.946 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747776+0000", 2024-09-15T21:01:14.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8e-07 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747806+0000", 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0182e-05 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747817+0000", 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0985000000000001e-05 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.847897+0000", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100080024 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948088+0000", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100191302 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948106+0000", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7869000000000001e-05 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949395+0000", 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001288909 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949456+0000", 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1086000000000004e-05 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 68/59 ECSubWrite(tid=234, reqid=osd.5.0:234, at_version=68'112, trim_to=0'0, roll_forward_to=68'110))", 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:00.504385+0000", 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.390624836000001, 2024-09-15T21:01:14.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104042788, 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504385+0000", 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504385+0000", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504387+0000", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9769999999999999e-06 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504433+0000", 2024-09-15T21:01:14.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.6023999999999998e-05 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504463+0000", 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0196e-05 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.504479+0000", 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6535999999999999e-05 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.604764+0000", 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10028461 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.604782+0000", 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7669999999999999e-05 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.606400+0000", 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001618677 2024-09-15T21:01:14.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:00.608427+0000", 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020270980000000002 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=236, reqid=osd.5.0:236, at_version=69'114, trim_to=0'0, roll_forward_to=68'112))", 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.843637139, 2024-09-15T21:01:14.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.201432585, 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051375+0000", 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8629999999999999e-06 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051452+0000", 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6847000000000005e-05 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051489+0000", 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7295999999999997e-05 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151633+0000", 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100143989 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151772+0000", 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138505 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151792+0000", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0516000000000001e-05 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252713+0000", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10092024300000001 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252805+0000", 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2325999999999997e-05 2024-09-15T21:01:14.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=238, reqid=osd.5.0:237, at_version=0'0, trim_to=0'0, roll_forward_to=69'116))", 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.436533585999999, 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202021808, 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458478+0000", 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2929999999999999e-06 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458507+0000", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9183000000000002e-05 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458535+0000", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7512999999999999e-05 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.558646+0000", 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100111397 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559059+0000", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041233799999999999 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559089+0000", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0145999999999999e-05 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660438+0000", 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134961000000001 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660498+0000", 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9327999999999998e-05 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=56, reqid=client.4641.0:576, at_version=69'122, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.184083232000001, 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30235248100000001, 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710927+0000", 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7000000000000001e-07 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711169+0000", 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242449 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711184+0000", 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4956999999999999e-05 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811338+0000", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015429200000001 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911614+0000", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027530699999999 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911632+0000", 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7963000000000001e-05 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.012893+0000", 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101261014 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013279+0000", 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038622899999999998 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.968 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=57, reqid=client.4641.0:577, at_version=69'123, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 13.083616409999999, 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20197553200000001, 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811394+0000", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1400000000000001e-06 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811565+0000", 2024-09-15T21:01:14.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017046300000000001 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811589+0000", 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4229000000000001e-05 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811601+0000", 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2194e-05 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912520+0000", 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100918439 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912530+0000", 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0720000000000001e-05 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013321+0000", 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100790989 2024-09-15T21:01:14.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013369+0000", 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7358000000000003e-05 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 75/59 ECSubWrite(tid=266, reqid=osd.5.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.1276611889999995, 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30105612199999998, 2024-09-15T21:01:14.973 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767349+0000", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.133e-06 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767364+0000", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4146e-05 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767381+0000", 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7144000000000001e-05 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.867460+0000", 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100079691 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967708+0000", 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100247187 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967719+0000", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0937e-05 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068375+0000", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100655967 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068404+0000", 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9916999999999998e-05 2024-09-15T21:01:14.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 78/59 ECSubWrite(tid=268, reqid=client.4641.0:643, at_version=78'141, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.9046825409999997, 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.203284137, 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990328+0000", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990551+0000", 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022275300000000001 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990576+0000", 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5414000000000001e-05 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.090713+0000", 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013612099999999 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.190988+0000", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027497 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.191018+0000", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0602000000000003e-05 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193517+0000", 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024985929999999999 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193611+0000", 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4324e-05 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 79/59 ECSubWrite(tid=277, reqid=client.4641.0:656, at_version=79'151, trim_to=0'0, roll_forward_to=78'145))", 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8773622730000001, 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.106015242, 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017649+0000", 2024-09-15T21:01:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2400000000000002e-06 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.018231+0000", 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00058144900000000005 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.018269+0000", 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8495000000000003e-05 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118372+0000", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100102858 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118538+0000", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016603200000000001 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118567+0000", 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8662000000000001e-05 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.123598+0000", 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050310440000000001 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.123662+0000", 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4461999999999994e-05 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:14.986 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:14.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:14.988 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:14.989 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:14.989 INFO:tasks.rados.rados.0.smithi027.stdout:296: finishing copy_from racing read to smithi02732947-5 2024-09-15T21:01:14.989 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:14.990 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 4 to smithi02732947-41 2024-09-15T21:01:14.990 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:14.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:14.983+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:15.005 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:15.005 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:15.033 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:15.033 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:15.059 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:15.082 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:15.082 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:15.089 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:15.083+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:15.126 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.126 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs1 81/28 ECSubWrite(tid=314, reqid=osd.5.0:311, at_version=0'0, trim_to=0'0, roll_forward_to=81'69))", 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.045787+0000", 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.079769686000000006, 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.079790480999999996, 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.045787+0000", 2024-09-15T21:01:15.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.045787+0000", 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.045787+0000", 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3799999999999998e-07 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.045797+0000", 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9299999999999998e-06 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.045807+0000", 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0122000000000001e-05 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:15.130 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T21:01:15.130 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:15.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:15.175 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 5 to smithi02732947-41 2024-09-15T21:01:15.175 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 6 to smithi02732947-41 2024-09-15T21:01:15.189 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:15.183+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:15.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.275+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:15.290 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:15.287+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:15.302 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:15.302 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:15.302 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:15.302 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:15.302 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:15.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:15.355 INFO:tasks.rados.rados.0.smithi027.stdout:296: finishing copy_from to smithi02732947-5 2024-09-15T21:01:15.355 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 5 v 21 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-15T21:01:15.380 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.375+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:01:15.384 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.379+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:15.384 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.379+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,6 to read 0,1,4,5 2024-09-15T21:01:15.384 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.379+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:01:15.385 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.379+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: finishing write tid 7 to smithi02732947-41 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 updating version 0 to 173 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 version 173 is already newer than 171 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 version 173 is already newer than 170 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 version 173 is already newer than 172 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 version 173 is already newer than 168 2024-09-15T21:01:15.386 INFO:tasks.rados.rados.0.smithi027.stdout:290: oid 41 updating version 173 to 174 2024-09-15T21:01:15.387 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 174 (ObjNum 122 snap 23 seq_num 122) dirty exists 2024-09-15T21:01:15.387 INFO:tasks.rados.rados.0.smithi027.stdout:290: left oid 41 (ObjNum 122 snap 23 seq_num 122) 2024-09-15T21:01:15.392 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:15.387+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:15.446 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 51.467517788000002, 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:01:15.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 51.465506542999996, 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:01:15.453 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:15.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 51.464754888999998, 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:01:15.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.465 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 51.464688705, 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.95942788, 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:15.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:01:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:01:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:01:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:01:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:01:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:15.479 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.957990930000001, 2024-09-15T21:01:15.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:01:15.480 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:15.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:01:15.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:01:15.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:01:15.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.957912784999998, 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.492 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:15.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:01:15.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:01:15.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:01:15.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:15.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:01:15.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:15.501 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.855397652000001, 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:01:15.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:01:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:15.507 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.775350924999998, 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:15.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:01:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.760962835000001, 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:15.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:15.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:01:15.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:01:15.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:01:15.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:15.523 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.754552879999999, 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:15.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:01:15.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:01:15.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.653999906999999, 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:01:15.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.572042529999997, 2024-09-15T21:01:15.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62454644199999998, 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871909+0000", 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871910+0000", 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.55e-06 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871958+0000", 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7698000000000003e-05 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.871989+0000", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1061e-05 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.872007+0000", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7934000000000001e-05 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496399+0000", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62439274499999997 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496455+0000", 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5454000000000001e-05 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.243376318000003, 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.538 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:15.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.140422368000003, 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66775810800000002, 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303530+0000", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8299999999999995e-07 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303918+0000", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038879600000000001 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303931+0000", 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-05 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303938+0000", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4359999999999999e-06 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971275+0000", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66733690499999998 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971287+0000", 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2007999999999999e-05 2024-09-15T21:01:15.546 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 35.133195782999998, 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66056677200000002, 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310756+0000", 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:15.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311135+0000", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037949099999999998 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311146+0000", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0736e-05 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311151+0000", 2024-09-15T21:01:15.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3299999999999998e-06 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971311+0000", 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66015986699999996 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971322+0000", 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.375405809, 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70767751899999998, 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068546+0000", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069555+0000", 2024-09-15T21:01:15.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010088269999999999 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069566+0000", 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0834e-05 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.169651+0000", 2024-09-15T21:01:15.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008542099999999 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776159+0000", 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60650789000000005 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776223+0000", 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3752999999999998e-05 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.278745411999999, 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71199342899999996, 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:15.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165208+0000", 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.88e-06 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166131+0000", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092355499999999995 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166158+0000", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7073e-05 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166172+0000", 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4249e-05 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877145+0000", 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71097216299999999 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877199+0000", 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4509000000000003e-05 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:671 3.2s0 3:48a96439:::smithi02732947-3:head [copy-from ver 128] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.5400064350000005, 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.82993921299999995, 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 671 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903948+0000", 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1370000000000002e-06 2024-09-15T21:01:15.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903963+0000", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4987000000000001e-05 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903998+0000", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4925000000000002e-05 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904016+0000", 2024-09-15T21:01:15.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8093000000000001e-05 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041691+0000", 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.13767560200000001 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041959+0000", 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026741900000000002 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:15.562 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.526650+0000", 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.484691182 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530502+0000", 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038516790000000002 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733738+0000", 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20323656000000001 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733884+0000", 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014562900000000001 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:672 3.2s0 3:48a96439:::smithi02732947-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.5399038679999997, 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62612606699999995, 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:15.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 672 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904048+0000", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904053+0000", 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7539999999999997e-06 2024-09-15T21:01:15.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904069+0000", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5866e-05 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904078+0000", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5529999999999998e-06 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.216745+0000", 2024-09-15T21:01:15.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31266638800000002 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.216844+0000", 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9193999999999999e-05 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.529855+0000", 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.313010538 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:15.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530002+0000", 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147515 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530173+0000", 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017116899999999999 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:15.569 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:15.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:15.571 INFO:tasks.rados.rados.0.smithi027.stdout:288: finishing copy_from racing read to smithi02732947-36 2024-09-15T21:01:15.572 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:15.567+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:290: done (5 left) 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:293: done (4 left) 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:296: done (3 left) 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:297: done (2 left) 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:298: read oid 46 snap -1 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:298: expect deleted 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 42 current snap is 24 2024-09-15T21:01:15.623 INFO:tasks.rados.rados.0.smithi027.stdout:299: seq_num 125 ranges {0=557056,557056=475136,1032192=507904,1540096=475136,2015232=98304} 2024-09-15T21:01:15.625 INFO:tasks.rados.rados.0.smithi027.stdout:299: writing smithi02732947-42 from 0 to 557056 tid 1 2024-09-15T21:01:15.628 INFO:tasks.rados.rados.0.smithi027.stdout:299: writing smithi02732947-42 from 557056 to 1032192 tid 2 2024-09-15T21:01:15.631 INFO:tasks.rados.rados.0.smithi027.stdout:299: writing smithi02732947-42 from 1032192 to 1540096 tid 3 2024-09-15T21:01:15.634 INFO:tasks.rados.rados.0.smithi027.stdout:299: writing smithi02732947-42 from 1540096 to 2015232 tid 4 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:299: writing smithi02732947-42 from 2015232 to 2113536 tid 5 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:300: read oid 34 snap -1 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:300: expect (ObjNum 123 snap 23 seq_num 123) 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:301: copy_from oid 29 from oid 2 current snap is 24 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 38 current snap is 24 2024-09-15T21:01:15.635 INFO:tasks.rados.rados.0.smithi027.stdout:302: seq_num 126 ranges {3276800=573440,3850240=622592,4472832=622592,5095424=311296} 2024-09-15T21:01:15.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:01:15.656 INFO:tasks.rados.rados.0.smithi027.stdout:302: writing smithi02732947-38 from 3276800 to 3850240 tid 1 2024-09-15T21:01:15.660 INFO:tasks.rados.rados.0.smithi027.stdout:302: writing smithi02732947-38 from 3850240 to 4472832 tid 2 2024-09-15T21:01:15.664 INFO:tasks.rados.rados.0.smithi027.stdout:302: writing smithi02732947-38 from 4472832 to 5095424 tid 3 2024-09-15T21:01:15.665 INFO:tasks.rados.rados.0.smithi027.stdout:302: writing smithi02732947-38 from 5095424 to 5406720 tid 4 2024-09-15T21:01:15.665 INFO:tasks.rados.rados.0.smithi027.stdout:303: snap_create 2024-09-15T21:01:15.672 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:15.667+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:15.714 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:15.714 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:15.725 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.719+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:15.726 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.719+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:15.729 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:15.740 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.735+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:15.743 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.739+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:15.743 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.739+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:15.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.767+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:15.804 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.799+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:15.804 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.799+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,4,5 2024-09-15T21:01:15.805 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.799+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,4,5 2024-09-15T21:01:15.805 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:15.799+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,7 to read 0,1,4,5 2024-09-15T21:01:15.827 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:15.827 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:15.828 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-15T21:01:15.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-15T21:01:15.829 INFO:tasks.rados.rados.0.smithi027.stdout:295: finishing copy_from racing read to smithi02732947-37 2024-09-15T21:01:15.839 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.835+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:15.839 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.835+0000 7fd53fdec640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:15.867 INFO:tasks.rados.rados.0.smithi027.stdout:288: finishing copy_from to smithi02732947-36 2024-09-15T21:01:15.867 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 36 v 166 (ObjNum 107 snap 18 seq_num 107) dirty exists 2024-09-15T21:01:15.869 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:15.869 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:15.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:15.867+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:15.883 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:15.934 INFO:tasks.rados.rados.0.smithi027.stdout:295: finishing copy_from to smithi02732947-37 2024-09-15T21:01:15.934 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 136 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-15T21:01:15.937 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.931+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:15.937 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:15.931+0000 7fd53bde4640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:15.951 INFO:tasks.rados.rados.0.smithi027.stdout:300: expect (ObjNum 123 snap 23 seq_num 123) 2024-09-15T21:01:15.965 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 2069991 2024-09-15T21:01:15.974 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:15.971+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:15.999 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:16.044 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:16.039+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:16.055 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.283729182000002, 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.282627816000002, 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:01:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:01:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.282587311999997, 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:01:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:01:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:01:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:01:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.262035803000003, 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:01:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:01:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.193748204999999, 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:01:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.160728947000003, 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:01:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:01:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.160688557, 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:01:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.092476392000002, 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.090573587000002, 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:01:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:01:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:01:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:01:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:01:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.088515258000001, 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202146044, 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:01:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965232+0000", 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965234+0000", 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7099999999999999e-06 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965259+0000", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5732999999999999e-05 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965292+0000", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2223e-05 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.965310+0000", 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8087000000000001e-05 2024-09-15T21:01:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167355+0000", 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202045846 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167378+0000", 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2444999999999999e-05 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.078591520000003, 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:01:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.077781780000002, 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:01:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.977850076000003, 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:01:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.958174045, 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.666732330000002, 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.469309264000003, 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:01:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:01:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.386073455999998, 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:01:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:01:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.693876986999999, 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:01:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 81/28 ECSubWriteReply(tid=37, last_complete=81'32, committed=1, applied=1))", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:14.827921+0000", 2024-09-15T21:01:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.2258258040000001, 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201444174, 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827921+0000", 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827921+0000", 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827923+0000", 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827932+0000", 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1169999999999993e-06 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827962+0000", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0122999999999999e-05 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.827978+0000", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5156e-05 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.028389+0000", 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20041091799999999 2024-09-15T21:01:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.029366+0000", 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097695000000000008 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 81/59 ECSubWrite(tid=256, reqid=osd.3.0:255, at_version=0'0, trim_to=0'0, roll_forward_to=81'90))", 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:14.828776+0000", 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.224971042, 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20067558099999999, 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.828776+0000", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.828776+0000", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.828778+0000", 2024-09-15T21:01:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1859999999999999e-06 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.828836+0000", 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7732999999999998e-05 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.828877+0000", 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0543000000000001e-05 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.928993+0000", 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100116365 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.929180+0000", 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000186808 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:14.929205+0000", 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5003000000000001e-05 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.029374+0000", 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100169234 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.029452+0000", 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7708999999999997e-05 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.151 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:16.153 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:16.143+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:16.153 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:16.143+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:16.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:16.154 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:16.147+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:16.159 INFO:tasks.ceph.mon.a.smithi027.stderr:2024-09-15T21:01:16.155+0000 7ff595055640 -1 mon.a@0(leader).osd e82 definitely_dead 0 2024-09-15T21:01:16.170 INFO:tasks.rados.rados.0.smithi027.stdout:301: finishing copy_from racing read to smithi02732947-29 2024-09-15T21:01:16.255 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.251+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:16.278 INFO:tasks.rados.rados.0.smithi027.stdout:301: finishing copy_from to smithi02732947-29 2024-09-15T21:01:16.278 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 29 v 76 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-15T21:01:16.319 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 1 to smithi02732947-42 2024-09-15T21:01:16.320 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 2 to smithi02732947-42 2024-09-15T21:01:16.338 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=325, last_complete=82'181, committed=1, applied=1))", 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.325519+0000", 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.011034577, 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011068866, 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325519+0000", 2024-09-15T21:01:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325519+0000", 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325520+0000", 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.133e-06 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325529+0000", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1939999999999998e-06 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325553+0000", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4199999999999999e-05 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325569+0000", 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5786000000000001e-05 2024-09-15T21:01:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=325, last_complete=82'181, committed=1, applied=1))", 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.327639+0000", 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0089152459999999999, 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0090229449999999992, 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327639+0000", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327639+0000", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327640+0000", 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1100000000000004e-07 2024-09-15T21:01:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327643+0000", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7390000000000001e-06 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327655+0000", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1904e-05 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327661+0000", 2024-09-15T21:01:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7570000000000001e-06 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=326, last_complete=82'182, committed=1, applied=1))", 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.327622+0000", 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0089318009999999996, 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0091029949999999991, 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327622+0000", 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327622+0000", 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327624+0000", 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7829999999999999e-06 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327635+0000", 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1212999999999999e-05 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327664+0000", 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8677e-05 2024-09-15T21:01:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327683+0000", 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9329000000000001e-05 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=325, last_complete=82'181, committed=1, applied=1))", 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.327913+0000", 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0086411539999999998, 2024-09-15T21:01:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0088748850000000008, 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327913+0000", 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327913+0000", 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327914+0000", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3859999999999999e-06 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327923+0000", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6139999999999992e-06 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327952+0000", 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8983e-05 2024-09-15T21:01:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.327973+0000", 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1495999999999999e-05 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=326, last_complete=82'182, committed=1, applied=1))", 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.328072+0000", 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0084816920000000007, 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0087759750000000001, 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328072+0000", 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328072+0000", 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328074+0000", 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.536e-06 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328082+0000", 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5269999999999998e-06 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328107+0000", 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5208999999999999e-05 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328124+0000", 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6099999999999998e-05 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 82/59 ECSubWriteReply(tid=326, last_complete=82'182, committed=1, applied=1))", 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.328230+0000", 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0083235129999999994, 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0086808019999999996, 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328230+0000", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328230+0000", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328231+0000", 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.68e-07 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328235+0000", 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7330000000000001e-06 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328244+0000", 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.328249+0000", 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1490000000000002e-06 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:764 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~507904 in=507904b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70534329799999995, 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70577652999999996, 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 764 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631213+0000", 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2170000000000001e-06 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631857+0000", 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00064422299999999997 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631889+0000", 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2246999999999998e-05 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631905+0000", 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5336e-05 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320301+0000", 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68839636000000004 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320584+0000", 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028351100000000002 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.322686+0000", 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021011860000000001 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.326185+0000", 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003499149 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:765 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~475136 in=475136b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70266142899999995, 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70320842299999997, 2024-09-15T21:01:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 765 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633894+0000", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2249999999999999e-06 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634374+0000", 2024-09-15T21:01:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048002400000000001 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634405+0000", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1189999999999998e-05 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634421+0000", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5815999999999999e-05 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324182+0000", 2024-09-15T21:01:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68976111799999995 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324311+0000", 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000129271 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325308+0000", 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099730799999999992 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:766 3.5s0 3.387730a5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70202466100000005, 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.702666444, 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 766 2024-09-15T21:01:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634531+0000", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4500000000000001e-06 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634708+0000", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000177104 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634721+0000", 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.359e-05 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634730+0000", 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9020000000000005e-06 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:767 3.5s0 3.387730a5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70180420099999996, 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70252350399999997, 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 767 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4499999999999996e-07 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634759+0000", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8650000000000007e-06 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634776+0000", 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6549e-05 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634783+0000", 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5340000000000002e-06 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:768 3.5s0 3.387730a5 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634796+0000", 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.70175738600000004, 2024-09-15T21:01:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70254990799999995, 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 768 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634796+0000", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634796+0000", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634797+0000", 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5899999999999995e-07 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634801+0000", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.783e-06 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634816+0000", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4975e-05 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634823+0000", 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.148e-06 2024-09-15T21:01:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 11 2024-09-15T21:01:16.378 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:16.380 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:16.380 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:16.375+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:16.455 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 1 to smithi02732947-38 2024-09-15T21:01:16.480 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:16.475+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:16.505 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:16.506 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:16.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:16.531 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 3 to smithi02732947-42 2024-09-15T21:01:16.555 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 2 to smithi02732947-38 2024-09-15T21:01:16.555 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 3 to smithi02732947-38 2024-09-15T21:01:16.555 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 4 to smithi02732947-38 2024-09-15T21:01:16.581 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.579+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:16.632 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:01:16.632 INFO:teuthology.orchestra.run.smithi027.stderr:marked in osd.1. 2024-09-15T21:01:16.637 INFO:tasks.rados.rados.0.smithi027.stdout:288: done (7 left) 2024-09-15T21:01:16.637 INFO:tasks.rados.rados.0.smithi027.stdout:295: done (6 left) 2024-09-15T21:01:16.637 INFO:tasks.rados.rados.0.smithi027.stdout:298: done (5 left) 2024-09-15T21:01:16.637 INFO:tasks.rados.rados.0.smithi027.stdout:300: done (4 left) 2024-09-15T21:01:16.637 INFO:tasks.rados.rados.0.smithi027.stdout:301: done (3 left) 2024-09-15T21:01:16.638 INFO:tasks.rados.rados.0.smithi027.stdout:303: done (2 left) 2024-09-15T21:01:16.638 INFO:tasks.rados.rados.0.smithi027.stdout:304: read oid 36 snap -1 2024-09-15T21:01:16.638 INFO:tasks.rados.rados.0.smithi027.stdout:304: expect (ObjNum 107 snap 18 seq_num 107) 2024-09-15T21:01:16.638 INFO:tasks.rados.rados.0.smithi027.stdout:305: delete oid 20 current snap is 25 2024-09-15T21:01:16.653 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:121 3.5s0 3:ac8a7abe:::smithi02732947-41:head [append 0~688128,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.488862363999999, 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82723362600000006, 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 121 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161394+0000", 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.161396+0000", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3630000000000001e-06 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162612+0000", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001215556 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162641+0000", 2024-09-15T21:01:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.904e-05 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162654+0000", 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.33e-05 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.262983+0000", 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100329088 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.263223+0000", 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023989200000000001 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.265887+0000", 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002663978 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369669+0000", 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10378166899999999 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988509+0000", 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61884030999999995 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.988628+0000", 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011843 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.487499946, 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:01:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:01:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:01:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.487457579999997, 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:01:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.487066652999999, 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:01:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.487027376, 2024-09-15T21:01:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.447795620999997, 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:01:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:01:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.447073269000001, 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:01:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:01:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.447056777999997, 2024-09-15T21:01:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:01:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:01:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:01:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:01:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.953869417999996, 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.947427953999998, 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:01:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:01:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 51.947001552000003, 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:01:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:01:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:01:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:350 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 36.507922643000001, 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82820832200000005, 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 350 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142334+0000", 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142335+0000", 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.717e-06 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142346+0000", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0869999999999999e-05 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142374+0000", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7322999999999999e-05 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142395+0000", 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1063000000000001e-05 2024-09-15T21:01:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142587+0000", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019266899999999999 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142604+0000", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6532999999999999e-05 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459842+0000", 2024-09-15T21:01:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31723833800000001 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.459989+0000", 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014666899999999999 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.970542+0000", 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51055313999999996 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:351 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 36.507808936000004, 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.83058971800000003, 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 351 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142447+0000", 2024-09-15T21:01:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142449+0000", 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.663e-06 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142459+0000", 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0083e-05 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142477+0000", 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7842999999999998e-05 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142492+0000", 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-05 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242885+0000", 2024-09-15T21:01:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100392596 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.242896+0000", 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1382999999999999e-05 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460558+0000", 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217662156 2024-09-15T21:01:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.460682+0000", 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124256 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.973037+0000", 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51235473799999998 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 36.507694313999998, 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:01:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:01:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:656 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~786432] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.7920273739999999, 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88797498799999997, 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 656 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858231+0000", 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.097e-06 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859347+0000", 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001116105 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859370+0000", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2289999999999998e-05 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859380+0000", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0771000000000001e-05 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.015842+0000", 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15646217000000001 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.016121+0000", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027814300000000001 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.017565+0000", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001443937 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.124892+0000", 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107327906 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.745992+0000", 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62109974199999995 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746204+0000", 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021182699999999999 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:657 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.7844841469999997, 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88093447300000005, 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 657 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865774+0000", 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866158+0000", 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038427300000000003 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866179+0000", 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0843999999999999e-05 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866190+0000", 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0628000000000001e-05 2024-09-15T21:01:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.018972+0000", 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.152781957 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.019140+0000", 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000168021 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.021442+0000", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023025670000000002 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.125095+0000", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.1036528 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746521+0000", 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62142534100000002 2024-09-15T21:01:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746707+0000", 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000186112 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:658 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~442368] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.7774449690000003, 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.87439944000000003, 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 658 2024-09-15T21:01:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872813+0000", 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2810000000000001e-06 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.873129+0000", 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031647199999999999 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.873148+0000", 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8893999999999999e-05 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973289+0000", 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100140643 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.022799+0000", 2024-09-15T21:01:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.049510586000000002 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.022941+0000", 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014148399999999999 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.023870+0000", 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092914500000000004 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.125098+0000", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101227734 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747009+0000", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62191137600000002 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747211+0000", 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020182500000000001 2024-09-15T21:01:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:661 3.5s0 3:a8fd5180:::smithi02732947-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6764125060000001, 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.979178514, 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 661 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973845+0000", 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973848+0000", 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3340000000000002e-06 2024-09-15T21:01:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973861+0000", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.34e-05 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973870+0000", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535782+0000", 2024-09-15T21:01:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56191254599999996 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535923+0000", 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140882 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748675+0000", 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21275177100000001 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748830+0000", 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000155682 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.953022+0000", 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20419203999999999 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:665 3.5s0 3:a4f26d94:::smithi02732947-27:head [append 0~425984] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.6763756159999996, 2024-09-15T21:01:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.87733781899999996, 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 665 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973882+0000", 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.974216+0000", 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033487799999999999 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.974243+0000", 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6738999999999999e-05 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.074353+0000", 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100109805 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.636146+0000", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56179291399999998 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.636411+0000", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026467999999999999 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.639540+0000", 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031295020000000002 2024-09-15T21:01:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747278+0000", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107738239 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.850998+0000", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103719222 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.851219+0000", 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022109399999999999 2024-09-15T21:01:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:764 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~507904 in=507904b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.019045862, 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89954056699999996, 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 764 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631213+0000", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2170000000000001e-06 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631857+0000", 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00064422299999999997 2024-09-15T21:01:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631889+0000", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2246999999999998e-05 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631905+0000", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5336e-05 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320301+0000", 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68839636000000004 2024-09-15T21:01:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320584+0000", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028351100000000002 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.322686+0000", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021011860000000001 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.326185+0000", 2024-09-15T21:01:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003499149 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530672+0000", 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20448741300000001 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530751+0000", 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8924999999999994e-05 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.778 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:16.779 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 5 to smithi02732947-38 2024-09-15T21:01:16.780 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.755+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:16.780 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.755+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:01:16.780 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.755+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:01:16.780 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.755+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,7 to read 0,1,4,5 2024-09-15T21:01:16.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:16.781 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-15T21:01:16.782 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 4 to smithi02732947-42 2024-09-15T21:01:16.782 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 5 to smithi02732947-42 2024-09-15T21:01:16.783 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:16.775+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:16.854 INFO:tasks.rados.rados.0.smithi027.stdout:305: done (3 left) 2024-09-15T21:01:16.854 INFO:tasks.rados.rados.0.smithi027.stdout:306: delete oid 15 current snap is 25 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: finishing write tid 6 to smithi02732947-38 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: oid 38 updating version 0 to 167 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: oid 38 updating version 167 to 169 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: oid 38 updating version 169 to 170 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: oid 38 updating version 170 to 171 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: oid 38 version 171 is already newer than 168 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 171 (ObjNum 126 snap 24 seq_num 126) dirty exists 2024-09-15T21:01:16.863 INFO:tasks.rados.rados.0.smithi027.stdout:302: left oid 38 (ObjNum 126 snap 24 seq_num 126) 2024-09-15T21:01:16.880 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:16.875+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.4s6 83/57 ECSubRead(tid=103, to_read={3:235c4434:::smithi02732947-36:head=0,1187840,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.861337+0000", 2024-09-15T21:01:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.057935644000000001, 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.057984734000000003, 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861337+0000", 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861337+0000", 2024-09-15T21:01:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861339+0000", 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.674e-06 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861357+0000", 2024-09-15T21:01:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8105999999999999e-05 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861403+0000", 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6309e-05 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.861432+0000", 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8197000000000001e-05 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:01:16.925 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:16.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:16.967 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.963+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:16.967 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.963+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:16.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.963+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:01:16.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.963+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:01:16.968 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.963+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4 to read 0,1,4,5 2024-09-15T21:01:16.977 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:16.971+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6 to read 0,1,4,5 2024-09-15T21:01:16.981 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:16.975+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:17.043 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:17.039+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:17.063 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:17.059+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:17.081 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:17.075+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:17.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:17.107 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:17.108 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:17.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:17.164 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:17.159+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:17.182 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:17.175+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.961338175999998, 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:01:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.959607683999998, 2024-09-15T21:01:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:01:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:01:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.958360106000001, 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:01:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.944476002000002, 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.860594001999999, 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.758147053999998, 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:01:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:01:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.757228155999996, 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.754704887000003, 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:01:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.752407849000001, 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:01:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:01:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.662811071999997, 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:01:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.659898392000002, 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:01:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 36.778349108999997, 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:01:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:01:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.0s2 53/28 ECSubWrite(tid=48, reqid=client.4641.0:387, at_version=53'12, trim_to=0'0, roll_forward_to=34'10))", 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.592115839000002, 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203297113, 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649672+0000", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649673+0000", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1199999999999999e-07 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649825+0000", 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015242899999999999 2024-09-15T21:01:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.649841+0000", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5931e-05 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.749979+0000", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100138358 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750181+0000", 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020151199999999999 2024-09-15T21:01:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.750213+0000", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1763000000000001e-05 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852931+0000", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102718235 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.852970+0000", 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8572999999999998e-05 2024-09-15T21:01:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.373227708000002, 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:01:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:502 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.601782743000001, 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30380249300000001, 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 502 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640008+0000", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8420000000000001e-06 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640020+0000", 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2067e-05 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640045+0000", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4406999999999999e-05 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640057+0000", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1917e-05 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640154+0000", 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6767000000000004e-05 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640165+0000", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.137e-05 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841240+0000", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20107475499999999 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841489+0000", 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024974399999999999 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.943808+0000", 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102318624 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:503 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.601709143000001, 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308825238, 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 503 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640080+0000", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4600000000000005e-07 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640085+0000", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9250000000000003e-06 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640092+0000", 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3019999999999997e-06 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640096+0000", 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1919999999999998e-06 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740388+0000", 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100292057 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740400+0000", 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1420000000000001e-05 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845387+0000", 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104987575 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845666+0000", 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000279158 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.948904+0000", 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103238063 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:504 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.601660631000001, 2024-09-15T21:01:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409270668, 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 504 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.46e-07 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640133+0000", 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4739999999999999e-06 2024-09-15T21:01:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640145+0000", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2673e-05 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640151+0000", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2829999999999998e-06 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740469+0000", 2024-09-15T21:01:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100317608 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740477+0000", 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2719999999999997e-06 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.846778+0000", 2024-09-15T21:01:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106300532 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.847035+0000", 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025722499999999998 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:57.049398+0000", 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20236315499999999 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 74/59 ECSubWrite(tid=259, reqid=client.4641.0:614, at_version=74'133, trim_to=0'0, roll_forward_to=73'130))", 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.539925737000001, 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20517745600000001, 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701865+0000", 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4870000000000001e-06 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702223+0000", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000357544 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702264+0000", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1263000000000001e-05 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802363+0000", 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10009968800000001 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802532+0000", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000168856 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802561+0000", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8552e-05 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.906960+0000", 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104399596 2024-09-15T21:01:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.907040+0000", 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9469999999999996e-05 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:770 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.6070191309999999, 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30194747500000002, 2024-09-15T21:01:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 770 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634770+0000", 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6300000000000004e-07 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634775+0000", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8219999999999997e-06 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634791+0000", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5994999999999998e-05 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734865+0000", 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10007408399999999 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735010+0000", 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014533800000000001 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735283+0000", 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027298399999999999 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.936717+0000", 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20143348899999999 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:771 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.506895841, 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21090594100000001, 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 771 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734894+0000", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.658e-06 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734901+0000", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2799999999999998e-06 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734910+0000", 2024-09-15T21:01:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7290000000000006e-06 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734915+0000", 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6859999999999997e-06 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.736101+0000", 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001186161 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.736208+0000", 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010704099999999999 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.945798+0000", 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20959038599999999 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:17.330 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:17.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:17.334 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:17.335 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:17.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:17.336 INFO:tasks.rados.rados.0.smithi027.stdout:304: expect (ObjNum 107 snap 18 seq_num 107) 2024-09-15T21:01:17.336 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:17.336 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:17.337 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:17.338 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:17.338 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:17.331+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:17.358 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:17.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:17.393 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:17.394 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:17.434 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:17.435 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:17.431+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:17.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:17.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:17.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:17.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:17.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:17.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:17.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:17.447+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,4,5 2024-09-15T21:01:17.469 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:17.479 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:17.480 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:17.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:17.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:17.504 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 6 to smithi02732947-42 2024-09-15T21:01:17.504 INFO:tasks.rados.rados.0.smithi027.stdout:302: done (3 left) 2024-09-15T21:01:17.505 INFO:tasks.rados.rados.0.smithi027.stdout:304: done (2 left) 2024-09-15T21:01:17.505 INFO:tasks.rados.rados.0.smithi027.stdout:306: done (1 left) 2024-09-15T21:01:17.505 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 37 current snap is 25 2024-09-15T21:01:17.505 INFO:tasks.rados.rados.0.smithi027.stdout:307: seq_num 129 ranges {4292608=622592,4915200=655360,5570560=688128,6258688=212992} 2024-09-15T21:01:17.518 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:17.518 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:17.538 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:17.531+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:17.543 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:17.547 INFO:tasks.rados.rados.0.smithi027.stdout:307: writing smithi02732947-37 from 4292608 to 4915200 tid 1 2024-09-15T21:01:17.554 INFO:tasks.rados.rados.0.smithi027.stdout:307: writing smithi02732947-37 from 4915200 to 5570560 tid 2 2024-09-15T21:01:17.560 INFO:tasks.rados.rados.0.smithi027.stdout:307: writing smithi02732947-37 from 5570560 to 6258688 tid 3 2024-09-15T21:01:17.562 INFO:tasks.rados.rados.0.smithi027.stdout:307: writing smithi02732947-37 from 6258688 to 6471680 tid 4 2024-09-15T21:01:17.562 INFO:tasks.rados.rados.0.smithi027.stdout:308: snap_create 2024-09-15T21:01:17.562 INFO:tasks.rados.rados.0.smithi027.stdout:299: finishing write tid 7 to smithi02732947-42 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 updating version 0 to 179 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 updating version 179 to 181 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 version 181 is already newer than 180 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 updating version 181 to 183 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 updating version 183 to 184 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: oid 42 version 184 is already newer than 182 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 184 (ObjNum 125 snap 24 seq_num 125) dirty exists 2024-09-15T21:01:17.563 INFO:tasks.rados.rados.0.smithi027.stdout:299: left oid 42 (ObjNum 125 snap 24 seq_num 125) 2024-09-15T21:01:17.573 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:17.640 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:17.635+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:17.645 INFO:tasks.rados.rados.0.smithi027.stdout:299: done (2 left) 2024-09-15T21:01:17.646 INFO:tasks.rados.rados.0.smithi027.stdout:308: done (1 left) 2024-09-15T21:01:17.646 INFO:tasks.rados.rados.0.smithi027.stdout:309: delete oid 35 current snap is 26 2024-09-15T21:01:17.663 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 1 to smithi02732947-37 2024-09-15T21:01:17.672 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:17.672 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:17.688 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:17.741 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:17.735+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:17.818 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:17.818 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:17.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:17.875 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:17.871+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:17.946 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:17.943+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:17.953 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 2 to smithi02732947-37 2024-09-15T21:01:17.953 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 3 to smithi02732947-37 2024-09-15T21:01:17.953 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 4 to smithi02732947-37 2024-09-15T21:01:17.965 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:17.965 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:17.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:18.053 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 5 to smithi02732947-37 2024-09-15T21:01:18.056 INFO:tasks.rados.rados.0.smithi027.stdout:309: done (1 left) 2024-09-15T21:01:18.056 INFO:tasks.rados.rados.0.smithi027.stdout:310: delete oid 6 current snap is 26 2024-09-15T21:01:18.058 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:18.051+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:18.066 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:18.063+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:18.144 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:18.145 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:18.150 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:18.143+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:18.153 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:18.147+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:01:18.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:18.250 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:18.247+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:18.258 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:18.255+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5 to read 0,1,4,5 2024-09-15T21:01:18.260 INFO:tasks.rados.rados.0.smithi027.stdout:307: finishing write tid 6 to smithi02732947-37 2024-09-15T21:01:18.260 INFO:tasks.rados.rados.0.smithi027.stdout:307: oid 37 updating version 0 to 138 2024-09-15T21:01:18.260 INFO:tasks.rados.rados.0.smithi027.stdout:307: oid 37 updating version 138 to 141 2024-09-15T21:01:18.260 INFO:tasks.rados.rados.0.smithi027.stdout:307: oid 37 version 141 is already newer than 139 2024-09-15T21:01:18.260 INFO:tasks.rados.rados.0.smithi027.stdout:307: oid 37 version 141 is already newer than 137 2024-09-15T21:01:18.261 INFO:tasks.rados.rados.0.smithi027.stdout:307: oid 37 version 141 is already newer than 140 2024-09-15T21:01:18.261 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 37 v 141 (ObjNum 129 snap 25 seq_num 129) dirty exists 2024-09-15T21:01:18.261 INFO:tasks.rados.rados.0.smithi027.stdout:307: left oid 37 (ObjNum 129 snap 25 seq_num 129) 2024-09-15T21:01:18.326 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:18.326 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:18.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:18.354 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:18.347+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:18.455 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:18.451+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:18.499 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:18.499 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:18.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:18.561 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:18.551+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:18.658 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:18.655+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:18.684 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:18.684 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:18.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:18.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:01:18.786 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:18.783+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:18.864 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:18.859+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:18.867 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:18.867 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:18.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:18.966 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:18.959+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:19.051 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.051 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.063 INFO:tasks.rados.rados.0.smithi027.stdout:307: done (1 left) 2024-09-15T21:01:19.063 INFO:tasks.rados.rados.0.smithi027.stdout:310: done (0 left) 2024-09-15T21:01:19.064 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 42 current snap is 26 2024-09-15T21:01:19.064 INFO:tasks.rados.rados.0.smithi027.stdout:311: seq_num 132 ranges {2113536=671744,2785280=573440,3358720=753664,4112384=229376} 2024-09-15T21:01:19.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:19.067 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:19.063+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:19.092 INFO:tasks.rados.rados.0.smithi027.stdout:311: writing smithi02732947-42 from 2113536 to 2785280 tid 1 2024-09-15T21:01:19.098 INFO:tasks.rados.rados.0.smithi027.stdout:311: writing smithi02732947-42 from 2785280 to 3358720 tid 2 2024-09-15T21:01:19.107 INFO:tasks.rados.rados.0.smithi027.stdout:311: writing smithi02732947-42 from 3358720 to 4112384 tid 3 2024-09-15T21:01:19.109 INFO:tasks.rados.rados.0.smithi027.stdout:311: writing smithi02732947-42 from 4112384 to 4341760 tid 4 2024-09-15T21:01:19.109 INFO:tasks.rados.rados.0.smithi027.stdout:312: snap_create 2024-09-15T21:01:19.169 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:19.163+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:19.190 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.190 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:19.270 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:19.263+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:19.307 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 1 to smithi02732947-42 2024-09-15T21:01:19.307 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 2 to smithi02732947-42 2024-09-15T21:01:19.370 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.370 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.371 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:19.367+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:19.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:19.471 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:19.467+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:19.512 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 3 to smithi02732947-42 2024-09-15T21:01:19.513 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 4 to smithi02732947-42 2024-09-15T21:01:19.549 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.549 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:19.572 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:19.567+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:19.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:19.614 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 5 to smithi02732947-42 2024-09-15T21:01:19.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:19.611+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:19.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:19.611+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,4,5 2024-09-15T21:01:19.649 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:01:19.654 INFO:tasks.rados.rados.0.smithi027.stdout:312: done (1 left) 2024-09-15T21:01:19.654 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 24 current snap is 27 2024-09-15T21:01:19.654 INFO:tasks.rados.rados.0.smithi027.stdout:313: seq_num 133 ranges {1835008=688128,2523136=720896,3244032=475136,3719168=360448} 2024-09-15T21:01:19.673 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:19.667+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:19.674 INFO:tasks.rados.rados.0.smithi027.stdout:313: writing smithi02732947-24 from 1835008 to 2523136 tid 1 2024-09-15T21:01:19.681 INFO:tasks.rados.rados.0.smithi027.stdout:313: writing smithi02732947-24 from 2523136 to 3244032 tid 2 2024-09-15T21:01:19.684 INFO:tasks.rados.rados.0.smithi027.stdout:313: writing smithi02732947-24 from 3244032 to 3719168 tid 3 2024-09-15T21:01:19.688 INFO:tasks.rados.rados.0.smithi027.stdout:313: writing smithi02732947-24 from 3719168 to 4079616 tid 4 2024-09-15T21:01:19.688 INFO:tasks.rados.rados.0.smithi027.stdout:314: rollback oid 49 current snap is 27 2024-09-15T21:01:19.688 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 49 to 25 2024-09-15T21:01:19.688 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 6 current snap is 27 2024-09-15T21:01:19.688 INFO:tasks.rados.rados.0.smithi027.stdout:315: seq_num 134 ranges {0=704512,704512=475136,1179648=573440,1753088=507904} 2024-09-15T21:01:19.691 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.692 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.694 INFO:tasks.rados.rados.0.smithi027.stdout:315: writing smithi02732947-6 from 0 to 704512 tid 1 2024-09-15T21:01:19.698 INFO:tasks.rados.rados.0.smithi027.stdout:315: writing smithi02732947-6 from 704512 to 1179648 tid 2 2024-09-15T21:01:19.702 INFO:tasks.rados.rados.0.smithi027.stdout:315: writing smithi02732947-6 from 1179648 to 1753088 tid 3 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:315: writing smithi02732947-6 from 1753088 to 2260992 tid 4 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:316: rmattr oid 13 current snap is 27 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:316: done (4 left) 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:317: read oid 18 snap -1 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:317: expect (ObjNum 120 snap 22 seq_num 120) 2024-09-15T21:01:19.706 INFO:tasks.rados.rados.0.smithi027.stdout:318: rollback oid 9 current snap is 27 2024-09-15T21:01:19.707 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 9 to 22 2024-09-15T21:01:19.707 INFO:tasks.rados.rados.0.smithi027.stdout:319: snap_create 2024-09-15T21:01:19.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:19.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:19.711+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6 to read 0,1,4,5 2024-09-15T21:01:19.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:19.711+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,4,5 2024-09-15T21:01:19.757 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:19.758 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:19.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:19.767+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:19.817 INFO:tasks.rados.rados.0.smithi027.stdout:311: finishing write tid 6 to smithi02732947-42 2024-09-15T21:01:19.817 INFO:tasks.rados.rados.0.smithi027.stdout:311: oid 42 updating version 0 to 197 2024-09-15T21:01:19.817 INFO:tasks.rados.rados.0.smithi027.stdout:311: oid 42 version 197 is already newer than 194 2024-09-15T21:01:19.818 INFO:tasks.rados.rados.0.smithi027.stdout:311: oid 42 updating version 197 to 198 2024-09-15T21:01:19.818 INFO:tasks.rados.rados.0.smithi027.stdout:311: oid 42 version 198 is already newer than 195 2024-09-15T21:01:19.818 INFO:tasks.rados.rados.0.smithi027.stdout:311: oid 42 version 198 is already newer than 196 2024-09-15T21:01:19.818 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 198 (ObjNum 132 snap 26 seq_num 132) dirty exists 2024-09-15T21:01:19.818 INFO:tasks.rados.rados.0.smithi027.stdout:311: left oid 42 (ObjNum 132 snap 26 seq_num 132) 2024-09-15T21:01:19.875 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:19.871+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:19.881 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:19.881 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:19.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:19.912 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:19.912 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:19.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:19.939 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:19.959 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:19.975 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:19.971+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:19.981 INFO:tasks.rados.rados.0.smithi027.stdout:314: finishing rollback tid 0 to smithi02732947-49 2024-09-15T21:01:19.982 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:19.983 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:20.009 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:20.009 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:20.035 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:20.035 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:20.036 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:20.036 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:20.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:20.095 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:20.091+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:20.178 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:20.171+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:20.224 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:20.225 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:20.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:20.279 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:20.275+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:20.315 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 1 to smithi02732947-6 2024-09-15T21:01:20.315 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 2 to smithi02732947-6 2024-09-15T21:01:20.315 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 3 to smithi02732947-6 2024-09-15T21:01:20.316 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 4 to smithi02732947-6 2024-09-15T21:01:20.316 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 5 to smithi02732947-6 2024-09-15T21:01:20.329 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:20.327+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:20.330 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:20.327+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:20.332 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:20.327+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:01:20.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:20.370 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:20.371 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:20.380 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.375+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:20.408 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:20.409 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:20.430 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 1 to smithi02732947-24 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: finishing write tid 6 to smithi02732947-6 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: oid 6 updating version 0 to 50 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: oid 6 updating version 50 to 51 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: oid 6 updating version 51 to 54 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: oid 6 version 54 is already newer than 53 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: oid 6 version 54 is already newer than 52 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 54 (ObjNum 134 snap 27 seq_num 134) dirty exists 2024-09-15T21:01:20.434 INFO:tasks.rados.rados.0.smithi027.stdout:315: left oid 6 (ObjNum 134 snap 27 seq_num 134) 2024-09-15T21:01:20.444 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:20.444 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:20.448 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:20.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.180384125000003, 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:01:20.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:01:20.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:20.454 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.180085986000002, 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:01:20.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:01:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:01:20.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.180061692000002, 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:20.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:01:20.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.972999229999999, 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:01:20.466 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:01:20.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.es4 32/28 ECSubWrite(tid=13, reqid=client.4641.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.897621016000002, 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32195022499999998, 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:20.471 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547348+0000", 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547350+0000", 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547377+0000", 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6630000000000001e-05 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547411+0000", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3708999999999999e-05 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.547429+0000", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867891+0000", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32046238700000002 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867902+0000", 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1385e-05 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869267+0000", 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013649020000000001 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.869298+0000", 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0997000000000002e-05 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.319763187, 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:20.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.479 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:01:20.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.482 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.219598128000001, 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.484 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:01:20.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 49.219553458, 2024-09-15T21:01:20.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.489 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:01:20.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:01:20.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:01:20.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.496 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.302081858999998, 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:01:20.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:01:20.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.301990472, 2024-09-15T21:01:20.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.504 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:01:20.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:359 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.138639083000001, 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35050568300000001, 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.510 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 359 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:20.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306332+0000", 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8339999999999999e-06 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306628+0000", 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029618500000000002 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.512 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306645+0000", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6518e-05 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306653+0000", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1960000000000008e-06 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306721+0000", 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7945000000000001e-05 2024-09-15T21:01:20.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306833+0000", 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112538 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.308213+0000", 2024-09-15T21:01:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001379915 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.309315+0000", 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001102107 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656634+0000", 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34731901900000001 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656836+0000", 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020142599999999999 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.138293312000002, 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.517 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:01:20.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.521 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:361 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~262144] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.037848281999999, 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35217888200000003, 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 361 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407122+0000", 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.567e-06 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407417+0000", 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029430100000000001 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407438+0000", 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1401000000000001e-05 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407450+0000", 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2036e-05 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409764+0000", 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023141680000000001 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409913+0000", 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000148777 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.411230+0000", 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00131726 2024-09-15T21:01:20.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413635+0000", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024047719999999999 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759187+0000", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34555227100000002 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759300+0000", 2024-09-15T21:01:20.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112329 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:362 3.6s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (56) in=7b] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.037497344999998, 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35221260100000001, 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.528 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 362 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407473+0000", 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407479+0000", 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5899999999999996e-06 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407492+0000", 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2435999999999999e-05 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407501+0000", 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5589999999999994e-06 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412116+0000", 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046143970000000001 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412250+0000", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013454100000000001 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413018+0000", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076778700000000003 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413954+0000", 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000935691 2024-09-15T21:01:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759576+0000", 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34562273999999998 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759684+0000", 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010788 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.037436548999999, 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:01:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:426 3.6s0 3:6d8f48d3:::smithi02732947-20:head [copy-from ver 70] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.189511937999999, 2024-09-15T21:01:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32441989700000001, 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 426 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255461+0000", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255475+0000", 2024-09-15T21:01:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.419e-05 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255507+0000", 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1507999999999997e-05 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255522+0000", 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5056e-05 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255654+0000", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013243 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255940+0000", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028538199999999999 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.470481+0000", 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21454129299999999 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.575970+0000", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105488665 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579610+0000", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003640636 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579877+0000", 2024-09-15T21:01:20.544 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000266894 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:674 3.6s0 3:6d8f48d3:::smithi02732947-20:head [read 152880~1370832,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.904127+0000", 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.5408418899999994, 2024-09-15T21:01:20.545 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32064719800000002, 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 674 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904127+0000", 2024-09-15T21:01:20.546 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904127+0000", 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904128+0000", 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5999999999999991e-07 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904134+0000", 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8669999999999999e-06 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904148+0000", 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4316e-05 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904157+0000", 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8799999999999997e-06 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.906225+0000", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0020673250000000001 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.906371+0000", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146439 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.224774+0000", 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31840341100000003 2024-09-15T21:01:20.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:675 3.6s0 3:6d8f48d3:::smithi02732947-20:head [read 618203~905509,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.5407560730000007, 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.323741204, 2024-09-15T21:01:20.550 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904214+0000", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6000000000000003e-07 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904219+0000", 2024-09-15T21:01:20.552 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0969999999999996e-06 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904231+0000", 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1613e-05 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904248+0000", 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7232999999999999e-05 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.553 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017097+0000", 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.11284968300000001 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017255+0000", 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015797900000000001 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.227954+0000", 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.210698839 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:772 3.1s0 3:81f89603:::smithi02732947-29:head [copy-from ver 76] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8103035949999997, 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.64200396500000001, 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.555 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634669+0000", 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4699999999999998e-06 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.557 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634686+0000", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7204999999999999e-05 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634729+0000", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3251999999999999e-05 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634756+0000", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6271e-05 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.558 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634873+0000", 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116989 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634882+0000", 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638177+0000", 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003295035 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.559 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638307+0000", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013013599999999999 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.165965+0000", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52765778900000004 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.171723+0000", 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057583169999999998 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276516+0000", 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104793367 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276669+0000", 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000153046 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.561 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:773 3.1s0 3:81f89603:::smithi02732947-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.8101601590000005, 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.53435976299999999, 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 773 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.562 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634811+0000", 2024-09-15T21:01:20.563 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.804e-06 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634818+0000", 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4549999999999998e-06 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634843+0000", 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4895000000000001e-05 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.564 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634866+0000", 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3198e-05 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634983+0000", 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000117088 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634995+0000", 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1912e-05 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.565 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638554+0000", 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00355902 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638636+0000", 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1588999999999994e-05 2024-09-15T21:01:20.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169005+0000", 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.530369117 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169073+0000", 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7940000000000003e-05 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169169+0000", 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5744999999999999e-05 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.568 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:20.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:20.570 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:20.571 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:20.571 INFO:tasks.rados.rados.0.smithi027.stdout:314: finishing rollback tid 1 to smithi02732947-49 2024-09-15T21:01:20.571 INFO:tasks.rados.rados.0.smithi027.stdout:314: finishing rollback tid 2 to smithi02732947-49 2024-09-15T21:01:20.571 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 49 v 94 (ObjNum 92 snap 14 seq_num 92) dirty exists 2024-09-15T21:01:20.572 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.491+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:20.572 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.491+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:20.572 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:20.573 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:20.573 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.495+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:01:20.574 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:20.563+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:20.589 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.583+0000 7f885aa8b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:20.591 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.587+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,7 to read 0,1,4,5 2024-09-15T21:01:20.592 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.587+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:20.593 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.587+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,4,5 2024-09-15T21:01:20.596 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.591+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:20.631 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 2 to smithi02732947-24 2024-09-15T21:01:20.632 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 3 to smithi02732947-24 2024-09-15T21:01:20.633 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 4 to smithi02732947-24 2024-09-15T21:01:20.633 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 5 to smithi02732947-24 2024-09-15T21:01:20.660 INFO:tasks.rados.rados.0.smithi027.stdout:311: done (6 left) 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:314: done (5 left) 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:315: done (4 left) 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:319: done (3 left) 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:320: rollback oid 41 current snap is 28 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 41 to 23 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:321: rollback oid 5 current snap is 28 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 5 to 25 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:322: rmattr oid 47 current snap is 28 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:322: done (5 left) 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:323: copy_from oid 14 from oid 22 current snap is 28 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 2 current snap is 28 2024-09-15T21:01:20.661 INFO:tasks.rados.rados.0.smithi027.stdout:324: seq_num 135 ranges {3260416=720896,3981312=622592,4603904=688128,5292032=245760} 2024-09-15T21:01:20.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:20.667+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:20.693 INFO:tasks.rados.rados.0.smithi027.stdout:324: writing smithi02732947-2 from 3260416 to 3981312 tid 1 2024-09-15T21:01:20.696 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.691+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:01:20.699 INFO:tasks.rados.rados.0.smithi027.stdout:324: writing smithi02732947-2 from 3981312 to 4603904 tid 2 2024-09-15T21:01:20.705 INFO:tasks.rados.rados.0.smithi027.stdout:324: writing smithi02732947-2 from 4603904 to 5292032 tid 3 2024-09-15T21:01:20.707 INFO:tasks.rados.rados.0.smithi027.stdout:324: writing smithi02732947-2 from 5292032 to 5537792 tid 4 2024-09-15T21:01:20.707 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 12 current snap is 28 2024-09-15T21:01:20.708 INFO:tasks.rados.rados.0.smithi027.stdout:325: seq_num 136 ranges {1638400=737280,2375680=770048,3145728=540672,3686400=245760} 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.2s5 87/84 [PushOp(3:4434bb98:::smithi02732947-5:head, version: 81'69, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:4434bb98:::smithi02732947-5:head@81'69, size: 1687552, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.662967+0000", 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.059779628000000001, 2024-09-15T21:01:20.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.059881517000000002, 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.662967+0000", 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.662967+0000", 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:20.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.662967+0000", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2799999999999996e-07 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.663198+0000", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023091 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.663229+0000", 2024-09-15T21:01:20.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1130999999999997e-05 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.663256+0000", 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6724e-05 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T21:01:20.727 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:20.728 INFO:tasks.rados.rados.0.smithi027.stdout:325: writing smithi02732947-12 from 1638400 to 2375680 tid 1 2024-09-15T21:01:20.731 INFO:tasks.rados.rados.0.smithi027.stdout:325: writing smithi02732947-12 from 2375680 to 3145728 tid 2 2024-09-15T21:01:20.735 INFO:tasks.rados.rados.0.smithi027.stdout:325: writing smithi02732947-12 from 3145728 to 3686400 tid 3 2024-09-15T21:01:20.737 INFO:tasks.rados.rados.0.smithi027.stdout:325: writing smithi02732947-12 from 3686400 to 3932160 tid 4 2024-09-15T21:01:20.737 INFO:tasks.rados.rados.0.smithi027.stdout:326: setattr oid 4 current snap is 28 2024-09-15T21:01:20.740 INFO:tasks.rados.rados.0.smithi027.stdout:327: snap_create 2024-09-15T21:01:20.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:20.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:20.739+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:20.773 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:20.767+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:20.826 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:20.819+0000 7f885ea93640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6,7 to read 0,1,4,5 2024-09-15T21:01:20.844 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:20.839+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:01:20.844 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:20.839+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,4,5 2024-09-15T21:01:20.844 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:20.839+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4,7 to read 0,1,4,5 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: finishing write tid 6 to smithi02732947-24 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: oid 24 updating version 0 to 74 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: oid 24 version 74 is already newer than 71 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: oid 24 version 74 is already newer than 72 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: oid 24 version 74 is already newer than 73 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: oid 24 updating version 74 to 75 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 24 v 75 (ObjNum 133 snap 27 seq_num 133) dirty exists 2024-09-15T21:01:20.846 INFO:tasks.rados.rados.0.smithi027.stdout:313: left oid 24 (ObjNum 133 snap 27 seq_num 133) 2024-09-15T21:01:20.874 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:20.867+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:20.914 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:20.915 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:20.915 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:20.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:20.975 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:20.971+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:21.031 INFO:tasks.rados.rados.0.smithi027.stdout:317: expect (ObjNum 120 snap 22 seq_num 120) 2024-09-15T21:01:21.031 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:21.075 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:21.071+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:21.106 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.345265566000002, 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.345218508000002, 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:01:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.345190707999997, 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.115 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.118 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.345165313999999, 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:01:21.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.122 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.039786092, 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.123 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:01:21.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.125 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.126 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.039225131999999, 2024-09-15T21:01:21.128 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.129 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.130 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.131 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.133 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.939066748999998, 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.134 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:01:21.136 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:21.137 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.138 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.938970279000003, 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.139 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.140 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.141 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:01:21.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.938012849000003, 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:01:21.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:01:21.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:01:21.148 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.149 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.937943343000001, 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:01:21.150 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:01:21.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:01:21.153 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:01:21.154 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.838175638000003, 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:01:21.155 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.156 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.158 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.734503465000003, 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.159 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.160 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.161 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:21.162 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.721243694000002, 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:21.163 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.165 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.719303267000001, 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.166 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:01:21.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:01:21.168 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.169 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.529649640000002, 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:01:21.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.120639210999997, 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:01:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.113117803000002, 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.920238193000003, 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:01:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.919235114999999, 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:01:21.187 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.25468231, 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:21.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.190 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.191 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:21.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:21.193 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:21.187+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:21.195 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 235369 2024-09-15T21:01:21.293 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.287+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:21.296 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:21.322 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:21.322 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:21.323 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:21.323 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:21.323 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:21.319+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:21.324 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:21.319+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:21.325 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:21.319+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,7 to read 0,1,4,5 2024-09-15T21:01:21.325 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:21.319+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:01:21.327 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:21.323+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,4,5 2024-09-15T21:01:21.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:21.394 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.391+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:318: finishing rollback tid 0 to smithi02732947-9 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 1 to smithi02732947-2 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 1 to smithi02732947-12 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 2 to smithi02732947-12 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 3 to smithi02732947-12 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 4 to smithi02732947-12 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 2 to smithi02732947-2 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 3 to smithi02732947-2 2024-09-15T21:01:21.401 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 4 to smithi02732947-2 2024-09-15T21:01:21.402 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 5 to smithi02732947-2 2024-09-15T21:01:21.402 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 55 (ObjNum 78 snap 10 seq_num 78) dirty exists 2024-09-15T21:01:21.417 INFO:tasks.rados.rados.0.smithi027.stdout:321: finishing rollback tid 0 to smithi02732947-5 2024-09-15T21:01:21.447 INFO:tasks.rados.rados.0.smithi027.stdout:320: finishing rollback tid 0 to smithi02732947-41 2024-09-15T21:01:21.447 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 5 to smithi02732947-12 2024-09-15T21:01:21.447 INFO:tasks.rados.rados.0.smithi027.stdout:320: finishing rollback tid 1 to smithi02732947-41 2024-09-15T21:01:21.447 INFO:tasks.rados.rados.0.smithi027.stdout:320: finishing rollback tid 2 to smithi02732947-41 2024-09-15T21:01:21.447 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 41 v 202 (ObjNum 122 snap 23 seq_num 122) dirty exists 2024-09-15T21:01:21.449 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:21.443+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:21.449 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:21.443+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,4,5 2024-09-15T21:01:21.461 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:21.461 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:21.461 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:21.461 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:21.461 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:21.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:21.495 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.491+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:21.518 INFO:tasks.rados.rados.0.smithi027.stdout:321: finishing rollback tid 1 to smithi02732947-5 2024-09-15T21:01:21.532 INFO:tasks.rados.rados.0.smithi027.stdout:321: finishing rollback tid 2 to smithi02732947-5 2024-09-15T21:01:21.532 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 5 v 131 (ObjNum 86 snap 14 seq_num 86) dirty exists 2024-09-15T21:01:21.533 INFO:tasks.rados.rados.0.smithi027.stdout:323: finishing copy_from racing read to smithi02732947-14 2024-09-15T21:01:21.551 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:21.547+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,7 to read 0,1,4,5 2024-09-15T21:01:21.579 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.575+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:21.579 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.575+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,4,5 2024-09-15T21:01:21.580 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.575+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,7 to read 0,1,4,5 2024-09-15T21:01:21.581 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:21.575+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,4,5 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: finishing write tid 6 to smithi02732947-2 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: oid 2 updating version 0 to 78 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: oid 2 version 78 is already newer than 77 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: oid 2 updating version 78 to 79 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: oid 2 updating version 79 to 80 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: oid 2 updating version 80 to 81 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 2 v 81 (ObjNum 135 snap 28 seq_num 135) dirty exists 2024-09-15T21:01:21.583 INFO:tasks.rados.rados.0.smithi027.stdout:324: left oid 2 (ObjNum 135 snap 28 seq_num 135) 2024-09-15T21:01:21.596 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:21.591+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.009298821999998, 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.610 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:01:21.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.521346432999998, 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:01:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:01:21.616 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.617 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 50.818332802999997, 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.618 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:21.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.620 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 48/28 ECSubRead(tid=132, to_read={3:a5da2594:::smithi02732947-25:head=0,315392,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.141815319000003, 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20088298299999999, 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:21.623 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.624 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463043+0000", 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463058+0000", 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4732999999999999e-05 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.563142+0000", 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008391699999999 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.625 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663346+0000", 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100204423 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663918+0000", 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057127899999999995 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.733791762999999, 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30162102800000001, 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:21.627 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871059+0000", 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.628 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871100+0000", 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0321999999999998e-05 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871128+0000", 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8208000000000002e-05 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871152+0000", 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4246000000000001e-05 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.629 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971464+0000", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312292 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971482+0000", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7176e-05 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172613+0000", 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20113102099999999 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172679+0000", 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6762999999999998e-05 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.733486903999999, 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20089554500000001, 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.632 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871364+0000", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871378+0000", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3706000000000001e-05 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871408+0000", 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0574999999999998e-05 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971547+0000", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013905400000001 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072019+0000", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100471385 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072259+0000", 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023990200000000001 2024-09-15T21:01:21.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.705853392000002, 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20140428699999999, 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.635 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898998+0000", 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4980000000000001e-06 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899107+0000", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010874699999999999 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899130+0000", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.268e-05 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999226+0000", 2024-09-15T21:01:21.637 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096764 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999366+0000", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139231 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999377+0000", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1097000000000001e-05 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100334+0000", 2024-09-15T21:01:21.638 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100957065 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100401+0000", 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7205000000000005e-05 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=180, reqid=client.4641.0:451, at_version=60'74, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:21.639 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 29.704074833, 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.200850169, 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.640 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900776+0000", 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.641 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900867+0000", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1712000000000006e-05 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900880+0000", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.218e-05 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001054+0000", 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10017451600000001 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.642 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001292+0000", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023806500000000001 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001309+0000", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101581+0000", 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027267099999999 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.643 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101625+0000", 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3965999999999999e-05 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 68/53 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=68'11, trim_to=0'0, roll_forward_to=67'9))", 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.857074588, 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20168053699999999, 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:21.645 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747776+0000", 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8e-07 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747806+0000", 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0182e-05 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.646 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747817+0000", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0985000000000001e-05 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.847897+0000", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100080024 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948088+0000", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100191302 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.647 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948106+0000", 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7869000000000001e-05 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949395+0000", 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001288909 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949456+0000", 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1086000000000004e-05 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=236, reqid=osd.5.0:236, at_version=69'114, trim_to=0'0, roll_forward_to=68'112))", 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.553477715, 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.201432585, 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051375+0000", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8629999999999999e-06 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051452+0000", 2024-09-15T21:01:21.650 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6847000000000005e-05 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051489+0000", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7295999999999997e-05 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151633+0000", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100143989 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151772+0000", 2024-09-15T21:01:21.651 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138505 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151792+0000", 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0516000000000001e-05 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252713+0000", 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10092024300000001 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.652 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252805+0000", 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2325999999999997e-05 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=238, reqid=osd.5.0:237, at_version=0'0, trim_to=0'0, roll_forward_to=69'116))", 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.146374162000001, 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202021808, 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.653 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.654 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458478+0000", 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2929999999999999e-06 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458507+0000", 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9183000000000002e-05 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458535+0000", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7512999999999999e-05 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.558646+0000", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100111397 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559059+0000", 2024-09-15T21:01:21.656 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041233799999999999 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559089+0000", 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0145999999999999e-05 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660438+0000", 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134961000000001 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660498+0000", 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9327999999999998e-05 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=56, reqid=client.4641.0:576, at_version=69'122, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.893923808, 2024-09-15T21:01:21.658 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30235248100000001, 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710927+0000", 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7000000000000001e-07 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.660 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711169+0000", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242449 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711184+0000", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4956999999999999e-05 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811338+0000", 2024-09-15T21:01:21.661 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015429200000001 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911614+0000", 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027530699999999 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911632+0000", 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7963000000000001e-05 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.012893+0000", 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101261014 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013279+0000", 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038622899999999998 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=57, reqid=client.4641.0:577, at_version=69'123, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.793456985999999, 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20197553200000001, 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811394+0000", 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1400000000000001e-06 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.665 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811565+0000", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017046300000000001 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811589+0000", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4229000000000001e-05 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811601+0000", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2194e-05 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.666 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912520+0000", 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100918439 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912530+0000", 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0720000000000001e-05 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013321+0000", 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100790989 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013369+0000", 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7358000000000003e-05 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 75/59 ECSubWrite(tid=266, reqid=osd.5.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 14.837501765000001, 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30105612199999998, 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767349+0000", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.133e-06 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767364+0000", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4146e-05 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767381+0000", 2024-09-15T21:01:21.670 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7144000000000001e-05 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.867460+0000", 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100079691 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967708+0000", 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100247187 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.671 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967719+0000", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0937e-05 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068375+0000", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100655967 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068404+0000", 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9916999999999998e-05 2024-09-15T21:01:21.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 78/59 ECSubWrite(tid=268, reqid=client.4641.0:643, at_version=78'141, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.614523116999999, 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.203284137, 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990328+0000", 2024-09-15T21:01:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990551+0000", 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022275300000000001 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990576+0000", 2024-09-15T21:01:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5414000000000001e-05 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.090713+0000", 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013612099999999 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.190988+0000", 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027497 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.191018+0000", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0602000000000003e-05 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193517+0000", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024985929999999999 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193611+0000", 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4324e-05 2024-09-15T21:01:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 79/59 ECSubWrite(tid=277, reqid=client.4641.0:656, at_version=79'151, trim_to=0'0, roll_forward_to=78'145))", 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.587202849000001, 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.106015242, 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.678 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017647+0000", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017649+0000", 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2400000000000002e-06 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.679 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.018231+0000", 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00058144900000000005 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.018269+0000", 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8495000000000003e-05 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.680 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118372+0000", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100102858 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118538+0000", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016603200000000001 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.118567+0000", 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8662000000000001e-05 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.681 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.123598+0000", 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050310440000000001 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.123662+0000", 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4461999999999994e-05 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 81/57 ECSubWrite(tid=93, reqid=client.4641.0:730, at_version=81'151, trim_to=0'0, roll_forward_to=81'150))", 2024-09-15T21:01:21.682 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 6.1789667689999996, 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.109107326, 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:21.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425886+0000", 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6400000000000001e-06 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431096+0000", 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052103649999999998 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431129+0000", 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2317000000000002e-05 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431145+0000", 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6651999999999999e-05 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.531373+0000", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100227812 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.531390+0000", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6818000000000001e-05 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.534901+0000", 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035111080000000002 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.534991+0000", 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9614e-05 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=130, reqid=client.4641.0:832, at_version=87'14, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.85356588, 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.188249429, 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751285+0000", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751828+0000", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054304400000000001 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751839+0000", 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751846+0000", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1110000000000002e-06 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836633+0000", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084786880999999995 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836646+0000", 2024-09-15T21:01:21.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2927e-05 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939450+0000", 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10280324 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939534+0000", 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4116000000000006e-05 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=131, reqid=client.4641.0:833, at_version=87'15, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.84792282299999999, 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.18273326100000001, 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756928+0000", 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-15T21:01:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757088+0000", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015978499999999999 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757098+0000", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0482999999999999e-05 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757103+0000", 2024-09-15T21:01:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2259999999999996e-06 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937980+0000", 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.180876436 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937987+0000", 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5440000000000001e-06 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939584+0000", 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001597137 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939661+0000", 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6057999999999996e-05 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=132, reqid=client.4641.0:834, at_version=87'16, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.84344909700000004, 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17882461699999999, 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761402+0000", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2500000000000005e-07 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761528+0000", 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012658 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761541+0000", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2294e-05 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761547+0000", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.117e-06 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938455+0000", 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.176908119 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938472+0000", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6898999999999999e-05 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940190+0000", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001718687 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940226+0000", 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5196e-05 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.701 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.701 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:21.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:01:21.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:21.647+0000 7f865695a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,5,7 to read 0,1,4,5 2024-09-15T21:01:21.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:21.704 INFO:tasks.rados.rados.0.smithi027.stdout:318: finishing rollback tid 1 to smithi02732947-9 2024-09-15T21:01:21.704 INFO:tasks.rados.rados.0.smithi027.stdout:318: finishing rollback tid 2 to smithi02732947-9 2024-09-15T21:01:21.704 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 79 (ObjNum 119 snap 22 seq_num 119) dirty exists 2024-09-15T21:01:21.704 INFO:tasks.rados.rados.0.smithi027.stdout:325: finishing write tid 6 to smithi02732947-12 2024-09-15T21:01:21.704 INFO:tasks.rados.rados.0.smithi027.stdout:325: oid 12 updating version 0 to 173 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: oid 12 updating version 173 to 175 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: oid 12 version 175 is already newer than 172 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: oid 12 version 175 is already newer than 174 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: oid 12 updating version 175 to 176 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 12 v 176 (ObjNum 136 snap 28 seq_num 136) dirty exists 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: left oid 12 (ObjNum 136 snap 28 seq_num 136) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:313: done (9 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:317: done (8 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:318: done (7 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:320: done (6 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:321: done (5 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:324: done (4 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:325: done (3 left) 2024-09-15T21:01:21.705 INFO:tasks.rados.rados.0.smithi027.stdout:326: done (2 left) 2024-09-15T21:01:21.706 INFO:tasks.rados.rados.0.smithi027.stdout:327: done (1 left) 2024-09-15T21:01:21.706 INFO:tasks.rados.rados.0.smithi027.stdout:328: copy_from oid 38 from oid 37 current snap is 29 2024-09-15T21:01:21.706 INFO:tasks.rados.rados.0.smithi027.stdout:329: snap_remove snap 26 2024-09-15T21:01:21.706 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:21.695+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:21.783 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:21.783 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:21.783 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-15T21:01:21.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:01:21.804 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:21.799+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:21.884 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:21.884 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 88/84 ECSubWriteReply(tid=289, last_complete=87'77, committed=1, applied=1))", 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.842163+0000", 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.040990683999999999, 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.041010321000000002, 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842163+0000", 2024-09-15T21:01:21.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842163+0000", 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842163+0000", 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2399999999999997e-07 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842168+0000", 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3939999999999998e-06 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842178+0000", 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8859999999999999e-06 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842183+0000", 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4280000000000004e-06 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 87/84 ECSubReadReply(tid=290, attrs_read=0))", 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.842716+0000", 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.040437596999999999, 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.040485330999999999, 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842716+0000", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842716+0000", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.842716+0000", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0200000000000001e-07 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843185+0000", 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000468135 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843196+0000", 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1018e-05 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843201+0000", 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5149999999999997e-06 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 87/84 ECSubWriteReply(tid=289, last_complete=87'77, committed=1, applied=1))", 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.843212+0000", 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.039941518000000002, 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.040012527999999999, 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843212+0000", 2024-09-15T21:01:21.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843212+0000", 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843212+0000", 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2099999999999998e-07 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843214+0000", 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3039999999999999e-06 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.843219+0000", 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5739999999999997e-06 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.893 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.3s1 88/28 ECSubRead(tid=49, to_read={3:cae12ce2:::smithi02732947-37:head=0,1617920,0}, subchunks={3:cae12ce2:::smithi02732947-37:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.869193+0000", 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.013960457000000001, 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.01405842, 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869193+0000", 2024-09-15T21:01:21.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869193+0000", 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869194+0000", 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5770000000000001e-06 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869207+0000", 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2147000000000001e-05 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869230+0000", 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3357e-05 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.869244+0000", 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4297999999999999e-05 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 88/84 ECSubReadReply(tid=290, attrs_read=0))", 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.840069+0000", 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.043084107000000003, 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.043204337000000002, 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:21.897 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840069+0000", 2024-09-15T21:01:21.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840069+0000", 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840070+0000", 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7299999999999995e-07 2024-09-15T21:01:21.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840511+0000", 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000441531 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840523+0000", 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1139e-05 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840528+0000", 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6579999999999999e-06 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 88/84 ECSubWriteReply(tid=289, last_complete=87'77, committed=1, applied=1))", 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:21.840543+0000", 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.042610782999999999, 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.042754320999999998, 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:21.901 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840543+0000", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840543+0000", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840543+0000", 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2100000000000002e-07 2024-09-15T21:01:21.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840546+0000", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840554+0000", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4540000000000007e-06 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.840561+0000", 2024-09-15T21:01:21.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8120000000000002e-06 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 6 2024-09-15T21:01:21.904 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:21.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:21.906 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:21.899+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:21.976 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:21.971+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:21.976 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:21.971+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:22.006 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:22.003+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:22.027 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:22.028 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:22.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:22.076 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:22.071+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4 to read 0,1,4,5 2024-09-15T21:01:22.107 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:22.103+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:22.134 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:22.135 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:01:22.153 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-15T21:01:21.661624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140149299,"num_objects":82,"num_object_clones":31,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":5,"num_read":423,"num_read_kb":117076,"num_write":822,"num_write_kb":163272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":66142227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":971,"ondisk_log_size":971,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":656844,"kb_used_data":282604,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1131805236,"statfs":{"total":1159641169920,"available":1158968561664,"internally_reserved":0,"allocated":289386496,"data_stored":367481528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,21,1,4,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25149440,"num_objects":12,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":17,"num_read_kb":32272,"num_write":28,"num_write_kb":24560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":13172736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019374"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":254,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.694805+0000","last_change":"2024-09-15T21:01:14.631539+0000","last_active":"2024-09-15T21:01:16.694805+0000","last_peered":"2024-09-15T21:01:16.694805+0000","last_clean":"2024-09-15T21:01:16.694805+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:01:16.694805+0000","last_undegraded":"2024-09-15T21:01:16.694805+0000","last_fullsized":"2024-09-15T21:01:16.694805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:42.287963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015562499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"3.e","version":"81'20","reported_seq":299,"reported_epoch":83,"state":"remapped+peering","last_fresh":"2024-09-15T21:01:16.648977+0000","last_change":"2024-09-15T21:01:17.649150+0000","last_active":"2024-09-15T21:01:16.648977+0000","last_peered":"2024-09-15T21:01:15.841647+0000","last_clean":"2024-09-15T21:01:15.841647+0000","last_became_active":"2024-09-15T21:01:07.916224+0000","last_became_peered":"2024-09-15T21:01:07.916224+0000","last_unstale":"2024-09-15T21:01:16.648977+0000","last_undegraded":"2024-09-15T21:01:16.648977+0000","last_fullsized":"2024-09-15T21:01:16.648977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30587246800000001,"stat_sum":{"num_bytes":786432,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":10,"num_read_kb":1257,"num_write":8,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,2147483647,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,4,6,8,9,10],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"3.d","version":"81'23","reported_seq":289,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.843451+0000","last_change":"2024-09-15T21:01:15.180915+0000","last_active":"2024-09-15T21:01:16.843451+0000","last_peered":"2024-09-15T21:01:16.843451+0000","last_clean":"2024-09-15T21:01:16.843451+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:01:16.843451+0000","last_undegraded":"2024-09-15T21:01:16.843451+0000","last_fullsized":"2024-09-15T21:01:16.843451+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:47.699060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.450640027,"stat_sum":{"num_bytes":1687552,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4922,"num_write":29,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"3.c","version":"81'69","reported_seq":325,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.646561+0000","last_change":"2024-09-15T21:01:15.038437+0000","last_active":"2024-09-15T21:01:16.646561+0000","last_peered":"2024-09-15T21:01:16.646561+0000","last_clean":"2024-09-15T21:01:16.646561+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:01:16.646561+0000","last_undegraded":"2024-09-15T21:01:16.646561+0000","last_fullsized":"2024-09-15T21:01:16.646561+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:20.759506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40897571900000002,"stat_sum":{"num_bytes":8028160,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":3103,"num_write":46,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"3.b","version":"86'94","reported_seq":410,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.824496+0000","last_change":"2024-09-15T21:01:14.829367+0000","last_active":"2024-09-15T21:01:20.824496+0000","last_peered":"2024-09-15T21:01:20.824496+0000","last_clean":"2024-09-15T21:01:20.824496+0000","last_became_active":"2024-09-15T21:00:51.891415+0000","last_became_peered":"2024-09-15T21:00:51.891415+0000","last_unstale":"2024-09-15T21:01:20.824496+0000","last_undegraded":"2024-09-15T21:01:20.824496+0000","last_fullsized":"2024-09-15T21:01:20.824496+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:39.956858+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.20993558800000001,"stat_sum":{"num_bytes":12288000,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":29767,"num_write":48,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"3.a","version":"81'15","reported_seq":291,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.650201+0000","last_change":"2024-09-15T21:01:15.038586+0000","last_active":"2024-09-15T21:01:16.650201+0000","last_peered":"2024-09-15T21:01:16.650201+0000","last_clean":"2024-09-15T21:01:16.650201+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:01:16.650201+0000","last_undegraded":"2024-09-15T21:01:16.650201+0000","last_fullsized":"2024-09-15T21:01:16.650201+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:30.197556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31160021700000001,"stat_sum":{"num_bytes":4423680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12576,"num_write":12,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"3.9","version":"81'69","reported_seq":394,"reported_epoch":87,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:20.661436+0000","last_change":"2024-09-15T21:01:18.969584+0000","last_active":"2024-09-15T21:01:20.661436+0000","last_peered":"2024-09-15T21:01:20.661436+0000","last_clean":"2024-09-15T21:01:15.739666+0000","last_became_active":"2024-09-15T21:01:18.958099+0000","last_became_peered":"2024-09-15T21:01:18.958099+0000","last_unstale":"2024-09-15T21:01:20.661436+0000","last_undegraded":"2024-09-15T21:01:18.547002+0000","last_fullsized":"2024-09-15T21:01:18.546025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.11977742199999999,"stat_sum":{"num_bytes":9011200,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":3262,"num_write":73,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":9371648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,2147483647,4,10],"avail_no_missing":["8(0)","4(6)","5(4)","6(5)","7(2)","9(3)","10(7)","11(1)"],"object_location_counts":[{"shards":"4(6),5(4),6(5),7(2),8(0),9(3),10(7),11(1)","objects":5}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"3.8","version":"79'12","reported_seq":290,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T21:01:17.747628+0000","last_change":"2024-09-15T21:01:14.934289+0000","last_active":"2024-09-15T21:01:17.747628+0000","last_peered":"2024-09-15T21:01:17.747628+0000","last_clean":"2024-09-15T21:01:17.747628+0000","last_became_active":"2024-09-15T21:00:45.947614+0000","last_became_peered":"2024-09-15T21:00:45.947614+0000","last_unstale":"2024-09-15T21:01:17.747628+0000","last_undegraded":"2024-09-15T21:01:17.747628+0000","last_fullsized":"2024-09-15T21:01:17.747628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.166286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20750359900000001,"stat_sum":{"num_bytes":3260416,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":5,"num_write_kb":3392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":180224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":259,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.843245+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:01:16.843245+0000","last_peered":"2024-09-15T21:01:16.843245+0000","last_clean":"2024-09-15T21:01:16.843245+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:01:16.843245+0000","last_undegraded":"2024-09-15T21:01:16.843245+0000","last_fullsized":"2024-09-15T21:01:16.843245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.643418+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:01:16.643418+0000","last_peered":"2024-09-15T21:01:16.643418+0000","last_clean":"2024-09-15T21:01:16.643418+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:01:16.643418+0000","last_undegraded":"2024-09-15T21:01:16.643418+0000","last_fullsized":"2024-09-15T21:01:16.643418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'77","reported_seq":436,"reported_epoch":87,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:21.532844+0000","last_change":"2024-09-15T21:01:21.532459+0000","last_active":"2024-09-15T21:01:21.532844+0000","last_peered":"2024-09-15T21:01:21.532844+0000","last_clean":"2024-09-15T21:01:15.726021+0000","last_became_active":"2024-09-15T21:01:18.761699+0000","last_became_peered":"2024-09-15T21:01:18.761699+0000","last_unstale":"2024-09-15T21:01:21.532844+0000","last_undegraded":"2024-09-15T21:01:21.531627+0000","last_fullsized":"2024-09-15T21:01:18.549943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.63767613499999998,"stat_sum":{"num_bytes":16908288,"num_objects":11,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":29,"num_read_kb":4403,"num_write":62,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":21315584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,2147483647,8,10],"avail_no_missing":["3(0)","4(3)","5(4)","6(2)","7(5)","8(6)","9(1)","10(7)"],"object_location_counts":[{"shards":"3(0),4(3),5(4),6(2),8(6),9(1),10(7)","objects":9}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.742868+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:01:16.742868+0000","last_peered":"2024-09-15T21:01:16.742868+0000","last_clean":"2024-09-15T21:01:16.742868+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:01:16.742868+0000","last_undegraded":"2024-09-15T21:01:16.742868+0000","last_fullsized":"2024-09-15T21:01:16.742868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:26.032254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'40","reported_seq":349,"reported_epoch":87,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:20.735679+0000","last_change":"2024-09-15T21:01:18.970382+0000","last_active":"2024-09-15T21:01:20.735679+0000","last_peered":"2024-09-15T21:01:20.735679+0000","last_clean":"2024-09-15T21:01:16.276520+0000","last_became_active":"2024-09-15T21:01:18.868498+0000","last_became_peered":"2024-09-15T21:01:18.868498+0000","last_unstale":"2024-09-15T21:01:20.735679+0000","last_undegraded":"2024-09-15T21:01:18.556266+0000","last_fullsized":"2024-09-15T21:01:18.552642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20939993800000001,"stat_sum":{"num_bytes":5865472,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":8035,"num_write":39,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5341184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,2147483647],"avail_no_missing":["0(0)","4(1)","5(4)","6(6)","7(7)","9(5)","10(2)","11(3)"],"object_location_counts":[{"shards":"0(0),4(1),5(4),6(6),7(7),9(5),10(2),11(3)","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.656631+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:01:16.656631+0000","last_peered":"2024-09-15T21:01:16.656631+0000","last_clean":"2024-09-15T21:01:16.656631+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:01:16.656631+0000","last_undegraded":"2024-09-15T21:01:16.656631+0000","last_fullsized":"2024-09-15T21:01:16.656631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"81'31","reported_seq":315,"reported_epoch":83,"state":"remapped+peering","last_fresh":"2024-09-15T21:01:16.647587+0000","last_change":"2024-09-15T21:01:17.748774+0000","last_active":"2024-09-15T21:01:16.647587+0000","last_peered":"2024-09-15T21:01:15.841090+0000","last_clean":"2024-09-15T21:01:15.841090+0000","last_became_active":"2024-09-15T21:01:08.863665+0000","last_became_peered":"2024-09-15T21:01:08.863665+0000","last_unstale":"2024-09-15T21:01:16.647587+0000","last_undegraded":"2024-09-15T21:01:16.647587+0000","last_fullsized":"2024-09-15T21:01:16.647587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.607917771,"stat_sum":{"num_bytes":7045120,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":9577,"num_write":55,"num_write_kb":6880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4308992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,2147483647,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,5,6,7,8,9],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":151,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-15T21:01:15.804219+0000","last_change":"2024-09-15T21:01:16.645078+0000","last_active":"2024-09-15T21:01:16.645078+0000","last_peered":"2024-09-15T21:01:15.804219+0000","last_clean":"2024-09-15T21:01:15.804219+0000","last_became_active":"2024-09-15T21:01:02.777523+0000","last_became_peered":"2024-09-15T21:01:02.777523+0000","last_unstale":"2024-09-15T21:01:15.804219+0000","last_undegraded":"2024-09-15T21:01:15.804219+0000","last_fullsized":"2024-09-15T21:01:15.804219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"84'42","reported_seq":311,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.741136+0000","last_change":"2024-09-15T21:01:15.130298+0000","last_active":"2024-09-15T21:01:20.741136+0000","last_peered":"2024-09-15T21:01:20.741136+0000","last_clean":"2024-09-15T21:01:20.741136+0000","last_became_active":"2024-09-15T21:00:20.410469+0000","last_became_peered":"2024-09-15T21:00:20.410469+0000","last_unstale":"2024-09-15T21:01:20.741136+0000","last_undegraded":"2024-09-15T21:01:20.741136+0000","last_fullsized":"2024-09-15T21:01:20.741136+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:23.774798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51149824399999999,"stat_sum":{"num_bytes":12189696,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":16,"num_read_kb":2318,"num_write":38,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"3.5","version":"82'183","reported_seq":512,"reported_epoch":83,"state":"active+clean+snaptrim","last_fresh":"2024-09-15T21:01:16.834640+0000","last_change":"2024-09-15T21:01:14.870483+0000","last_active":"2024-09-15T21:01:16.834640+0000","last_peered":"2024-09-15T21:01:16.834640+0000","last_clean":"2024-09-15T21:01:16.834640+0000","last_became_active":"2024-09-15T21:00:51.794785+0000","last_became_peered":"2024-09-15T21:00:51.794785+0000","last_unstale":"2024-09-15T21:01:16.834640+0000","last_undegraded":"2024-09-15T21:01:16.834640+0000","last_fullsized":"2024-09-15T21:01:16.834640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:46.646433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.7609380720000001,"stat_sum":{"num_bytes":28409856,"num_objects":14,"num_object_clones":6,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":63,"num_read_kb":6500,"num_write":139,"num_write_kb":34336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12353536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"5"},{"start":"17","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.937207+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:01:20.937207+0000","last_peered":"2024-09-15T21:01:20.937207+0000","last_clean":"2024-09-15T21:01:20.937207+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:01:20.937207+0000","last_undegraded":"2024-09-15T21:01:20.937207+0000","last_fullsized":"2024-09-15T21:01:20.937207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:10.241744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"82'157","reported_seq":432,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T21:01:17.665502+0000","last_change":"2024-09-15T21:01:15.239142+0000","last_active":"2024-09-15T21:01:17.665502+0000","last_peered":"2024-09-15T21:01:17.665502+0000","last_clean":"2024-09-15T21:01:17.665502+0000","last_became_active":"2024-09-15T21:00:50.048868+0000","last_became_peered":"2024-09-15T21:00:50.048868+0000","last_unstale":"2024-09-15T21:01:17.665502+0000","last_undegraded":"2024-09-15T21:01:17.665502+0000","last_fullsized":"2024-09-15T21:01:17.665502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:46.818840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.54572884399999999,"stat_sum":{"num_bytes":24838144,"num_objects":11,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":45,"num_read_kb":23997,"num_write":82,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[10,11,9,5,7,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"e","length":"1"},{"start":"10","length":"6"},{"start":"17","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.760474+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:01:20.760474+0000","last_peered":"2024-09-15T21:01:20.760474+0000","last_clean":"2024-09-15T21:01:20.760474+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:01:20.760474+0000","last_undegraded":"2024-09-15T21:01:20.760474+0000","last_fullsized":"2024-09-15T21:01:20.760474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"34'5","reported_seq":304,"reported_epoch":87,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:20.658236+0000","last_change":"2024-09-15T21:01:19.160831+0000","last_active":"2024-09-15T21:01:20.658236+0000","last_peered":"2024-09-15T21:01:20.658236+0000","last_clean":"2024-09-15T21:01:15.826039+0000","last_became_active":"2024-09-15T21:01:19.058048+0000","last_became_peered":"2024-09-15T21:01:19.058048+0000","last_unstale":"2024-09-15T21:01:20.658236+0000","last_undegraded":"2024-09-15T21:01:18.551959+0000","last_fullsized":"2024-09-15T21:01:18.550012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063942700000000003,"stat_sum":{"num_bytes":1490944,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3466,"num_write":5,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,2147483647],"avail_no_missing":["3(0)","0(3)","4(6)","5(7)","7(2)","9(5)","10(4)","11(1)"],"object_location_counts":[{"shards":"0(3),3(0),4(6),5(7),7(2),9(5),10(4),11(1)","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.760736+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:01:20.760736+0000","last_peered":"2024-09-15T21:01:20.760736+0000","last_clean":"2024-09-15T21:01:20.760736+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:01:20.760736+0000","last_undegraded":"2024-09-15T21:01:20.760736+0000","last_fullsized":"2024-09-15T21:01:20.760736+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"83'39","reported_seq":345,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T21:01:20.836712+0000","last_change":"2024-09-15T21:01:14.938389+0000","last_active":"2024-09-15T21:01:20.836712+0000","last_peered":"2024-09-15T21:01:20.836712+0000","last_clean":"2024-09-15T21:01:20.836712+0000","last_became_active":"2024-09-15T21:00:48.749851+0000","last_became_peered":"2024-09-15T21:00:48.749851+0000","last_unstale":"2024-09-15T21:01:20.836712+0000","last_undegraded":"2024-09-15T21:01:20.836712+0000","last_fullsized":"2024-09-15T21:01:20.836712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:45.217681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20849875400000001,"stat_sum":{"num_bytes":3325952,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":23,"num_read_kb":3828,"num_write":56,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2473984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-15T21:01:16.649588+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:01:16.649588+0000","last_peered":"2024-09-15T21:01:16.649588+0000","last_clean":"2024-09-15T21:01:16.649588+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:01:16.649588+0000","last_undegraded":"2024-09-15T21:01:16.649588+0000","last_fullsized":"2024-09-15T21:01:16.649588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:08.004258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":139558912,"num_objects":78,"num_object_clones":31,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":5,"num_read":347,"num_read_kb":117012,"num_write":697,"num_write_kb":161120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":66142208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282525696,"data_stored":374398976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378644,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":16700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":17100800,"data_stored":20661142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411348,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61528,"kb_used_data":30340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94310312,"statfs":{"total":96636764160,"available":96573759488,"internally_reserved":0,"allocated":31068160,"data_stored":38472045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59628,"kb_used_data":28440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312212,"statfs":{"total":96636764160,"available":96575705088,"internally_reserved":0,"allocated":29122560,"data_stored":34878358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53216,"kb_used_data":22032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318624,"statfs":{"total":96636764160,"available":96582270976,"internally_reserved":0,"allocated":22560768,"data_stored":29964995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60740,"kb_used_data":29556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311100,"statfs":{"total":96636764160,"available":96574566400,"internally_reserved":0,"allocated":30265344,"data_stored":36971958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,6,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444052,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58712,"kb_used_data":27524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94313128,"statfs":{"total":96636764160,"available":96576643072,"internally_reserved":0,"allocated":28184576,"data_stored":36572412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,0,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444052,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62800,"kb_used_data":31604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309040,"statfs":{"total":96636764160,"available":96572456960,"internally_reserved":0,"allocated":32362496,"data_stored":39207849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444051,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61572,"kb_used_data":30384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94310268,"statfs":{"total":96636764160,"available":96573714432,"internally_reserved":0,"allocated":31113216,"data_stored":37286825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,14,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444051,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48648,"kb_used_data":17464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323192,"statfs":{"total":96636764160,"available":96586948608,"internally_reserved":0,"allocated":17883136,"data_stored":23157443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444053,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56484,"kb_used_data":25296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315356,"statfs":{"total":96636764160,"available":96578924544,"internally_reserved":0,"allocated":25903104,"data_stored":40312060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444051,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1911510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444052,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52200,"kb_used_data":21016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319640,"statfs":{"total":96636764160,"available":96583311360,"internally_reserved":0,"allocated":21520384,"data_stored":28084931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21024768,"data_stored":30167040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3747840,"data_stored":3747840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16134144,"data_stored":20066304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25817088,"data_stored":39337984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17444864,"data_stored":24764416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30142464,"data_stored":36691968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31391744,"data_stored":38612992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28704768,"data_stored":35786752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":22122496,"data_stored":32587776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":27738112,"data_stored":39395328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28700672,"data_stored":35901440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":29556736,"data_stored":37339136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:01:22.155 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-15T21:01:22.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-15T21:01:22.169 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:22.169 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.190354126000003, 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:01:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:01:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.174 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.175 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.188342880999997, 2024-09-15T21:01:22.176 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.177 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:01:22.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.179 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.180 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:01:22.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.182 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:119 3.bs0 3:d87a9ad1:::smithi02732947-40:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.187591226999999, 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68955062899999997, 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.183 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 119 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979196+0000", 2024-09-15T21:01:22.184 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979197+0000", 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979203+0000", 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.6919999999999999e-06 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979215+0000", 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979244+0000", 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9329e-05 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.186 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182534+0000", 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20328970900000001 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.182705+0000", 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017179700000000001 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.187 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183317+0000", 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00061195900000000003 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183921+0000", 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00060378000000000005 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668500+0000", 2024-09-15T21:01:22.188 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48457924699999999 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668747+0000", 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024636800000000001 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.189 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.187525043000001, 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.190 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.191 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:01:22.192 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:01:22.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.194 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.682264218, 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:01:22.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:01:22.198 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:01:22.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.200 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:01:22.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.680827268000002, 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.204 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:01:22.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.206 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:01:22.207 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.208 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.680749122999998, 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.209 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.210 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:01:22.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.212 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:01:22.213 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:22.214 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.578233990000001, 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.215 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:01:22.216 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.498187262999998, 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62577947099999998, 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.219 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668600+0000", 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.220 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668601+0000", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2240000000000001e-06 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668620+0000", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8848e-05 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668639+0000", 2024-09-15T21:01:22.221 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668655+0000", 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5498e-05 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294198+0000", 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.625543089 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.294380+0000", 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018193800000000001 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.483799173000001, 2024-09-15T21:01:22.223 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.224 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:01:22.225 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.227 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.229 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.477389217999999, 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:01:22.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.233 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:01:22.234 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.236 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.376836245, 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.237 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:22.238 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.240 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:01:22.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.966212656000003, 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:01:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:01:22.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.250 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.251 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.252 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.863258706000003, 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66775810800000002, 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303529+0000", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303530+0000", 2024-09-15T21:01:22.254 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8299999999999995e-07 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303918+0000", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038879600000000001 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303931+0000", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.308e-05 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.303938+0000", 2024-09-15T21:01:22.255 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4359999999999999e-06 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971275+0000", 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66733690499999998 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971287+0000", 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2007999999999999e-05 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.256 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.bs0 48/28 ECSubReadReply(tid=114, attrs_read=0))", 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.856032120999998, 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66056677200000002, 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310755+0000", 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.310756+0000", 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:22.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311135+0000", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00037949099999999998 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311146+0000", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0736e-05 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.311151+0000", 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3299999999999998e-06 2024-09-15T21:01:22.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971311+0000", 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.66015986699999996 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971322+0000", 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.098242147000001, 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70767751899999998, 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068546+0000", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069555+0000", 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010088269999999999 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069566+0000", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0834e-05 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.169651+0000", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008542099999999 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776159+0000", 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60650789000000005 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776223+0000", 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3752999999999998e-05 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.00158175, 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71199342899999996, 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165208+0000", 2024-09-15T21:01:22.265 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.88e-06 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166131+0000", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092355499999999995 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166158+0000", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7073e-05 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166172+0000", 2024-09-15T21:01:22.266 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4249e-05 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877145+0000", 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71097216299999999 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877199+0000", 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4509000000000003e-05 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:671 3.2s0 3:48a96439:::smithi02732947-3:head [copy-from ver 128] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.262842772999999, 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.82993921299999995, 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 671 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903948+0000", 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1370000000000002e-06 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903963+0000", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4987000000000001e-05 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903998+0000", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4925000000000002e-05 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904016+0000", 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8093000000000001e-05 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041691+0000", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.13767560200000001 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041959+0000", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026741900000000002 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.526650+0000", 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.484691182 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530502+0000", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038516790000000002 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733738+0000", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20323656000000001 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733884+0000", 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014562900000000001 2024-09-15T21:01:22.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:672 3.2s0 3:48a96439:::smithi02732947-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.262740206, 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.62612606699999995, 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 672 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904047+0000", 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.274 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904048+0000", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904053+0000", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7539999999999997e-06 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904069+0000", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5866e-05 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.275 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904078+0000", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5529999999999998e-06 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.216745+0000", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31266638800000002 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.216844+0000", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9193999999999999e-05 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.276 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.529855+0000", 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.313010538 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530002+0000", 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000147515 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530173+0000", 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017116899999999999 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:830 3.2s0 3:447dff51:::smithi02732947-14:head [copy-from ver 48] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.251715141, 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.92164203099999997, 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 830 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:22.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915073+0000", 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915079+0000", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7559999999999996e-06 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915093+0000", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.366e-05 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915101+0000", 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6959999999999994e-06 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316410+0000", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40130822100000002 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316742+0000", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033288600000000001 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.525877+0000", 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20913427800000001 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.634058+0000", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.108180927 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836242+0000", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20218430800000001 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836714+0000", 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047230599999999997 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:22.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:22.283 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:22.284 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:22.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:22.279+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:22.385 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:22.379+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:22.420 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:22.420 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:22.420 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:22.420 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:22.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:22.486 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:22.483+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:22.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:22.587 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:22.583+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:22.611 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:22.612 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:22.612 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:22.612 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:22.612 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:22.626 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:01:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 3 pg_num to 26 2024-09-15T21:01:22.666 INFO:tasks.rados.rados.0.smithi027.stdout:329: done (2 left) 2024-09-15T21:01:22.666 INFO:tasks.rados.rados.0.smithi027.stdout:330: rollback oid 8 current snap is 29 2024-09-15T21:01:22.666 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 8 to 28 2024-09-15T21:01:22.666 INFO:tasks.rados.rados.0.smithi027.stdout:331: read oid 3 snap -1 2024-09-15T21:01:22.666 INFO:tasks.rados.rados.0.smithi027.stdout:331: expect (ObjNum 105 snap 17 seq_num 105) 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:332: snap_create 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:323: finishing copy_from to smithi02732947-14 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 14 v 48 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:328: finishing copy_from racing read to smithi02732947-38 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:328: finishing copy_from to smithi02732947-38 2024-09-15T21:01:22.667 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 38 v 141 (ObjNum 129 snap 25 seq_num 129) dirty exists 2024-09-15T21:01:22.688 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:22.683+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:22.789 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:22.783+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:22.796 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.023061587000001, 2024-09-15T21:01:22.797 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.798 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.799 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.800 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:01:22.801 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.802 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.803 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.021960221000001, 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:01:22.804 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:01:22.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.806 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:22.807 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.807 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.807 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:01:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:01:22.808 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:01:22.809 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.810 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.811 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.021919717000003, 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:22.812 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.813 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.814 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.815 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:01:22.816 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.001368208000002, 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:01:22.817 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.818 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.819 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:01:22.820 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:01:22.821 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.822 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 32/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.933080609999998, 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20491661999999999, 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.823 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.859999+0000", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860000+0000", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.178e-06 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860007+0000", 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4339999999999997e-06 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.824 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860024+0000", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7887999999999999e-05 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.860036+0000", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1822e-05 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064854+0000", 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20481745100000001 2024-09-15T21:01:22.825 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.064916+0000", 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1847000000000006e-05 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.900061352000002, 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:01:22.826 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.827 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.828 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:22.828 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.828 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.828 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:01:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.830 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:22.831 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:22.832 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.900020961999999, 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:22.833 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.834 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.835 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:01:22.836 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.837 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.3s0 32/28 ECSubRead(tid=3, to_read={3:c82167fc:::smithi02732947-32:head=0,4096,0}, subchunks={3:c82167fc:::smithi02732947-32:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.831808797000001, 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20456293, 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.838 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.839 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.839 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:22.839 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.839 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.840 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.840 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961271+0000", 2024-09-15T21:01:22.841 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961990+0000", 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00071883299999999999 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962018+0000", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8713000000000001e-05 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165425+0000", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20340643 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.165834+0000", 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000408954 2024-09-15T21:01:22.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.829905992, 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20414806599999999, 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.844 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963174+0000", 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.845 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963175+0000", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.243e-06 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963201+0000", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5953e-05 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963219+0000", 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.874e-05 2024-09-15T21:01:22.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.963231+0000", 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1742999999999999e-05 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.166721+0000", 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203490114 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.847 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.167322+0000", 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00060027300000000002 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.817923925000002, 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.848 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.849 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:01:22.850 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:01:22.851 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.817114185000001, 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:22.853 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:01:22.854 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.855 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:22.856 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.717182481000002, 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:22.857 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.860 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:01:22.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=2, last_complete=32'2, committed=1, applied=1))", 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.697506449999999, 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27270873000000001, 2024-09-15T21:01:22.862 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.863 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095573+0000", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6399999999999998e-07 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095576+0000", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7269999999999999e-06 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.095589+0000", 2024-09-15T21:01:22.864 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3174e-05 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.195685+0000", 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100095935 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368245+0000", 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.172559557 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.865 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368282+0000", 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7073000000000002e-05 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 33/28 ECSubWriteReply(tid=7, last_complete=33'6, committed=1, applied=1))", 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.406064735000001, 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27975661299999999, 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.866 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387015+0000", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.867 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387016+0000", 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2999999999999999e-07 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387021+0000", 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8550000000000001e-06 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387036+0000", 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5271999999999999e-05 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.868 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.387044+0000", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5820000000000002e-06 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666718+0000", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.279673475 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.666771+0000", 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.3498999999999997e-05 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.869 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.es3 33/28 ECSubWrite(tid=17, reqid=client.4641.0:142, at_version=33'3, trim_to=0'0, roll_forward_to=31'2))", 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.208641669000002, 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20255242900000001, 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.870 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584438+0000", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584439+0000", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.871 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584777+0000", 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033844000000000002 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584796+0000", 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8865999999999999e-05 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.584807+0000", 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0421000000000001e-05 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.872 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684986+0000", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100179041 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.684995+0000", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.4690000000000003e-06 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786907+0000", 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101912261 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.873 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.786990+0000", 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2933e-05 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.125405860999997, 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.875 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.876 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:01:22.877 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:01:22.878 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.433209392000002, 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.879 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:01:22.880 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.881 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:01:22.882 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.10.18:4 3.3s0 3:cae12ce2:::smithi02732947-37:head [assert-version v141,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.028592937, 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32170995400000002, 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.883 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764489+0000", 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.593e-06 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764500+0000", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0463e-05 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764521+0000", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.162e-05 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764537+0000", 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5603e-05 2024-09-15T21:01:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764721+0000", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018443800000000001 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764741+0000", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9346000000000001e-05 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868783+0000", 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10404185000000001 2024-09-15T21:01:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868926+0000", 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014331200000000001 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086197+0000", 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217270729 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 88/28 ECSubReadReply(tid=49, attrs_read=0))", 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:21.876118+0000", 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.91696142000000003, 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.210691196, 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876118+0000", 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876118+0000", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876120+0000", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9640000000000002e-06 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.880776+0000", 2024-09-15T21:01:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0046557919999999997 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.880815+0000", 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8713e-05 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.880836+0000", 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1333000000000001e-05 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.890 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086774+0000", 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20593850799999999 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086809+0000", 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4885999999999998e-05 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 87/28 ECSubReadReply(tid=49, attrs_read=0))", 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:21.876767+0000", 2024-09-15T21:01:22.891 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.91631208099999994, 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.209903642, 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876767+0000", 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876767+0000", 2024-09-15T21:01:22.892 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.876855+0000", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7996000000000002e-05 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.881603+0000", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0047477780000000002 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.881629+0000", 2024-09-15T21:01:22.893 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5553000000000001e-05 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.881641+0000", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1796e-05 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086602+0000", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20496138 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086671+0000", 2024-09-15T21:01:22.894 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9139000000000003e-05 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:22.895 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:22.897 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:22.875+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:22.897 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:22.875+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:22.897 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:22.875+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4 to read 0,1,4,5 2024-09-15T21:01:22.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:22.898 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:22.891+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:22.985 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:22.985 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:22.997 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:22.991+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:23.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:23.006 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:23.030 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 89/28 ECSubWriteReply(tid=360, last_complete=89'83, committed=1, applied=1))", 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:22.964540+0000", 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.064978751000000001, 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.065016176999999994, 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.031 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.964540+0000", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.964540+0000", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.964541+0000", 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6099999999999999e-07 2024-09-15T21:01:23.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.964547+0000", 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1099999999999999e-06 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.964559+0000", 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143999999999999e-05 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.033 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 89/59 ECSubWriteReply(tid=361, last_complete=89'204, committed=1, applied=1))", 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:22.965078+0000", 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.064440203000000001, 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.064543884999999995, 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965078+0000", 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965078+0000", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965079+0000", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2699999999999998e-07 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965084+0000", 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.04e-06 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965100+0000", 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5279999999999999e-05 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.965108+0000", 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9289999999999998e-06 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.036 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 89/59 ECSubWriteReply(tid=361, last_complete=89'204, committed=1, applied=1))", 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:22.968871+0000", 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.060647630000000001, 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.060816368000000003, 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968871+0000", 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968871+0000", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968873+0000", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6869999999999999e-06 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968884+0000", 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0806e-05 2024-09-15T21:01:23.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968908+0000", 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4862999999999999e-05 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.968922+0000", 2024-09-15T21:01:23.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3297e-05 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s4 89/84 ECSubRead(tid=298, to_read={3:48a96439:::smithi02732947-3:head=0,499712,0}, subchunks={3:48a96439:::smithi02732947-3:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:22.978079+0000", 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.051439383999999998, 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.051682883999999998, 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:23.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.978079+0000", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.978079+0000", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.978080+0000", 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:01:23.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.978098+0000", 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7609999999999999e-05 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.978344+0000", 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024574699999999999 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:23.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 89/59 ECSubWriteReply(tid=361, last_complete=89'204, committed=1, applied=1))", 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:22.979050+0000", 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.050468478999999997, 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.050766955000000002, 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979050+0000", 2024-09-15T21:01:23.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979050+0000", 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979052+0000", 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1469999999999999e-06 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:23.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979061+0000", 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6330000000000002e-06 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979086+0000", 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4859e-05 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.979098+0000", 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1939999999999999e-05 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.045 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:23.046 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.046 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:23.046 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:23.046 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 5 2024-09-15T21:01:23.046 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:23.047 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:23.083 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.075+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:23.083 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.075+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:23.083 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.075+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:23.099 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.095+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:23.101 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:23.101 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:23.128 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:23.157 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:23.157 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:23.169 INFO:tasks.rados.rados.0.smithi027.stdout:330: finishing rollback tid 0 to smithi02732947-8 2024-09-15T21:01:23.188 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.183+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:23.188 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.183+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3 to read 0,1,4,5 2024-09-15T21:01:23.188 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:23.183+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:23.192 INFO:tasks.rados.rados.0.smithi027.stdout:331: expect (ObjNum 105 snap 17 seq_num 105) 2024-09-15T21:01:23.192 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:23.199 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:23.195+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:23.210 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:23.210 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:23.220 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:23.222 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:23.222 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:23.237 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:23.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:23.254 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 380712 2024-09-15T21:01:23.273 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:23.273 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:23.300 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:23.295+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:23.303 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:23.303 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:23.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:23.333 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:23.362 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:23.362 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:23.368 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:23.368 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:23.384 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:23.394 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:23.427 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.423+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:23.503 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:23.499+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:323: done (4 left) 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:328: done (3 left) 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:331: done (2 left) 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:332: done (1 left) 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 4 current snap is 30 2024-09-15T21:01:23.548 INFO:tasks.rados.rados.0.smithi027.stdout:333: seq_num 137 ranges {2310144=753664,3063808=786432,3850240=655360,4505600=114688} 2024-09-15T21:01:23.550 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:23.550 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:23.566 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:23.582 INFO:tasks.rados.rados.0.smithi027.stdout:333: writing smithi02732947-4 from 2310144 to 3063808 tid 1 2024-09-15T21:01:23.590 INFO:tasks.rados.rados.0.smithi027.stdout:333: writing smithi02732947-4 from 3063808 to 3850240 tid 2 2024-09-15T21:01:23.597 INFO:tasks.rados.rados.0.smithi027.stdout:333: writing smithi02732947-4 from 3850240 to 4505600 tid 3 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:333: writing smithi02732947-4 from 4505600 to 4620288 tid 4 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:334: rmattr oid 6 current snap is 30 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:334: done (2 left) 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:335: read oid 1 snap -1 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:335: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 22 current snap is 30 2024-09-15T21:01:23.599 INFO:tasks.rados.rados.0.smithi027.stdout:336: seq_num 138 ranges {3735552=770048,4505600=540672,5046272=753664,5799936=262144} 2024-09-15T21:01:23.610 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:23.607+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:23.635 INFO:tasks.rados.rados.0.smithi027.stdout:336: writing smithi02732947-22 from 3735552 to 4505600 tid 1 2024-09-15T21:01:23.639 INFO:tasks.rados.rados.0.smithi027.stdout:336: writing smithi02732947-22 from 4505600 to 5046272 tid 2 2024-09-15T21:01:23.643 INFO:tasks.rados.rados.0.smithi027.stdout:336: writing smithi02732947-22 from 5046272 to 5799936 tid 3 2024-09-15T21:01:23.645 INFO:tasks.rados.rados.0.smithi027.stdout:336: writing smithi02732947-22 from 5799936 to 6062080 tid 4 2024-09-15T21:01:23.646 INFO:tasks.rados.rados.0.smithi027.stdout:337: read oid 29 snap -1 2024-09-15T21:01:23.646 INFO:tasks.rados.rados.0.smithi027.stdout:337: expect (ObjNum 87 snap 14 seq_num 87) 2024-09-15T21:01:23.646 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 44 current snap is 30 2024-09-15T21:01:23.646 INFO:tasks.rados.rados.0.smithi027.stdout:338: seq_num 139 ranges {0=786432,786432=688128,1474560=475136,1949696=393216} 2024-09-15T21:01:23.650 INFO:tasks.rados.rados.0.smithi027.stdout:338: writing smithi02732947-44 from 0 to 786432 tid 1 2024-09-15T21:01:23.654 INFO:tasks.rados.rados.0.smithi027.stdout:338: writing smithi02732947-44 from 786432 to 1474560 tid 2 2024-09-15T21:01:23.657 INFO:tasks.rados.rados.0.smithi027.stdout:338: writing smithi02732947-44 from 1474560 to 1949696 tid 3 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:338: writing smithi02732947-44 from 1949696 to 2342912 tid 4 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:339: rollback oid 43 current snap is 30 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 43 to 28 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:340: read oid 9 snap -1 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:340: expect (ObjNum 119 snap 22 seq_num 119) 2024-09-15T21:01:23.660 INFO:tasks.rados.rados.0.smithi027.stdout:341: snap_remove snap 23 2024-09-15T21:01:23.665 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.659+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:23.665 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.659+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:23.698 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:23.698 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:23.705 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:23.699+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:23.713 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:23.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.755+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:23.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.755+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:23.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.755+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6 to read 0,1,4,5 2024-09-15T21:01:23.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:23.799+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:23.859 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.851+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:23.859 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.851+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:23.859 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:23.860 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:01:23.860 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:23.860 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:23.860 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:01:23.861 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.855+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:23.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.859+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:01:23.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.863+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,4,5 2024-09-15T21:01:23.867 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.863+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:01:23.867 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.863+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3 to read 0,1,4,5 2024-09-15T21:01:23.867 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.863+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,4,5 2024-09-15T21:01:23.875 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:23.871+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,7 to read 0,1,4,5 2024-09-15T21:01:23.886 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:23.887 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:23.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:23.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:23.899+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:24.008 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.003+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:24.060 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:24.055+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5,6 to read 0,1,4,5 2024-09-15T21:01:24.076 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.076 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.091 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:24.139 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:24.135+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:24.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.179+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:24.211 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:24.207+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:24.220 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.221 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.235 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:24.272 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:24.267+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:24.273 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:24.267+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:24.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.283+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:24.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.283+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:01:24.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.283+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,7 to read 0,1,4,5 2024-09-15T21:01:24.312 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:24.307+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:24.373 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:24.366+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4 to read 0,1,4,5 2024-09-15T21:01:24.374 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:24.366+0000 7f5a7b9ca640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,4 to read 0,1,4,5 2024-09-15T21:01:24.405 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.405 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.414 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.407+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:24.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:24.515 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:24.511+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:24.553 INFO:tasks.rados.rados.0.smithi027.stdout:341: done (8 left) 2024-09-15T21:01:24.553 INFO:tasks.rados.rados.0.smithi027.stdout:342: read oid 20 snap -1 2024-09-15T21:01:24.553 INFO:tasks.rados.rados.0.smithi027.stdout:342: expect deleted 2024-09-15T21:01:24.553 INFO:tasks.rados.rados.0.smithi027.stdout:343: snap_create 2024-09-15T21:01:24.554 INFO:tasks.rados.rados.0.smithi027.stdout:339: finishing rollback tid 0 to smithi02732947-43 2024-09-15T21:01:24.554 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 1 to smithi02732947-22 2024-09-15T21:01:24.554 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 2 to smithi02732947-22 2024-09-15T21:01:24.554 INFO:tasks.rados.rados.0.smithi027.stdout:340: expect (ObjNum 119 snap 22 seq_num 119) 2024-09-15T21:01:24.554 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:24.557 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.557 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:24.616 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:24.611+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:24.620 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:24.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:01:24.700 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.700 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.707 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 3843484 2024-09-15T21:01:24.713 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 3 to smithi02732947-22 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 1 to smithi02732947-44 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 2 to smithi02732947-44 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 4 to smithi02732947-22 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 3 to smithi02732947-44 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 4 to smithi02732947-44 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 5 to smithi02732947-44 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 5 to smithi02732947-22 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 1 to smithi02732947-4 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 2 to smithi02732947-4 2024-09-15T21:01:24.714 INFO:tasks.rados.rados.0.smithi027.stdout:336: finishing write tid 6 to smithi02732947-22 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: oid 22 updating version 0 to 73 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: oid 22 version 73 is already newer than 72 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: oid 22 updating version 73 to 75 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: oid 22 version 75 is already newer than 74 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: oid 22 version 75 is already newer than 71 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 22 v 75 (ObjNum 138 snap 30 seq_num 138) dirty exists 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:336: left oid 22 (ObjNum 138 snap 30 seq_num 138) 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: finishing write tid 6 to smithi02732947-44 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: oid 44 updating version 0 to 209 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: oid 44 version 209 is already newer than 207 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: oid 44 version 209 is already newer than 208 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: oid 44 version 209 is already newer than 205 2024-09-15T21:01:24.715 INFO:tasks.rados.rados.0.smithi027.stdout:338: oid 44 version 209 is already newer than 206 2024-09-15T21:01:24.716 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 44 v 209 (ObjNum 139 snap 30 seq_num 139) dirty exists 2024-09-15T21:01:24.716 INFO:tasks.rados.rados.0.smithi027.stdout:338: left oid 44 (ObjNum 139 snap 30 seq_num 139) 2024-09-15T21:01:24.716 INFO:tasks.rados.rados.0.smithi027.stdout:337: expect (ObjNum 87 snap 14 seq_num 87) 2024-09-15T21:01:24.716 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:24.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:24.718 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:24.711+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:24.754 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:24.797 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:24.818 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:24.815+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:24.837 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.837 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.843 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 3 to smithi02732947-4 2024-09-15T21:01:24.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:24.858 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.851+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:24.859 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.851+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:01:24.859 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.851+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,4,5 2024-09-15T21:01:24.863 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.855+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:24.863 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.855+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:24.863 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.855+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:24.863 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.859+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:24.922 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:24.915+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:24.964 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:24.959+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:24.974 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:24.974 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:24.981 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 4 to smithi02732947-4 2024-09-15T21:01:24.981 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 5 to smithi02732947-4 2024-09-15T21:01:24.983 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:24.978+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:24.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:25.034 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:25.030+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:25.078 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:25.070+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:25.083 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:25.078+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4,7 to read 0,1,4,5 2024-09-15T21:01:25.115 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.115 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.126 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:25.118+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:25.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: finishing write tid 6 to smithi02732947-4 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: oid 4 updating version 0 to 57 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: oid 4 updating version 57 to 59 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: oid 4 version 59 is already newer than 58 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: oid 4 version 59 is already newer than 56 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: oid 4 updating version 59 to 60 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 60 (ObjNum 137 snap 30 seq_num 137) dirty exists 2024-09-15T21:01:25.179 INFO:tasks.rados.rados.0.smithi027.stdout:333: left oid 4 (ObjNum 137 snap 30 seq_num 137) 2024-09-15T21:01:25.185 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:25.178+0000 7f84bf513640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 1c after trimming was started 2024-09-15T21:01:25.227 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:25.223+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:25.268 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:25.259+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:25.296 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.297 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:25.328 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:25.323+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:25.379 INFO:tasks.rados.rados.0.smithi027.stdout:335: expect (ObjNum 95 snap 17 seq_num 95) 2024-09-15T21:01:25.406 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 2835357 2024-09-15T21:01:25.417 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:25.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:25.463 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:25.459+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:25.475 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:25.477 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.477 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:25.530 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:25.523+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:01:25.555 INFO:tasks.rados.rados.0.smithi027.stdout:333: done (9 left) 2024-09-15T21:01:25.555 INFO:tasks.rados.rados.0.smithi027.stdout:335: done (8 left) 2024-09-15T21:01:25.555 INFO:tasks.rados.rados.0.smithi027.stdout:336: done (7 left) 2024-09-15T21:01:25.555 INFO:tasks.rados.rados.0.smithi027.stdout:337: done (6 left) 2024-09-15T21:01:25.555 INFO:tasks.rados.rados.0.smithi027.stdout:338: done (5 left) 2024-09-15T21:01:25.556 INFO:tasks.rados.rados.0.smithi027.stdout:340: done (4 left) 2024-09-15T21:01:25.556 INFO:tasks.rados.rados.0.smithi027.stdout:342: done (3 left) 2024-09-15T21:01:25.556 INFO:tasks.rados.rados.0.smithi027.stdout:343: done (2 left) 2024-09-15T21:01:25.556 INFO:tasks.rados.rados.0.smithi027.stdout:344: delete oid 38 current snap is 31 2024-09-15T21:01:25.571 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:25.571 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:25.588 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:25.588 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:25.604 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:25.613 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.613 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:25.631 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:25.626+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:25.681 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:25.681 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:25.707 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:25.732 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:25.726+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:25.734 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:25.734 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:25.751 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.752 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.762 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:25.762 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:25.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:25.767 INFO:tasks.rados.rados.0.smithi027.stdout:344: done (2 left) 2024-09-15T21:01:25.767 INFO:tasks.rados.rados.0.smithi027.stdout:345: rollback oid 6 current snap is 31 2024-09-15T21:01:25.767 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 6 to 25 2024-09-15T21:01:25.767 INFO:tasks.rados.rados.0.smithi027.stdout:346: rmattr oid 16 current snap is 31 2024-09-15T21:01:25.768 INFO:tasks.rados.rados.0.smithi027.stdout:347: snap_remove snap 30 2024-09-15T21:01:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:25.792 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:25.815 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:25.815 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:25.833 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:25.827+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:25.893 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:25.923 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:25.934 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:25.927+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:25.945 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:25.945 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:25.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:26.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:26.031+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:26.083 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:26.083 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:26.098 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:26.124 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:26.125 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:26.147 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:26.142+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:26.226 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.639407726999998, 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.227 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:01:26.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.638700370000002, 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:26.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:01:26.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.638127226999998, 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.238 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:26.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:01:26.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.537885285999998, 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.244 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:01:26.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:01:26.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.431626985999998, 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:01:26.251 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:01:26.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:01:26.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:01:26.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:01:26.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:26.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.431514550999999, 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:01:26.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:01:26.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.417557180999999, 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38400288500000002, 2024-09-15T21:01:26.265 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804716+0000", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804722+0000", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0689999999999998e-06 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804740+0000", 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7978999999999998e-05 2024-09-15T21:01:26.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904872+0000", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100132263 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188682+0000", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28381031600000001 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188718+0000", 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:01:26.269 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.781391313, 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39968706100000001, 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.270 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440882+0000", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440886+0000", 2024-09-15T21:01:26.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440911+0000", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494999999999998e-05 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440924+0000", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3142999999999999e-05 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840527+0000", 2024-09-15T21:01:26.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39960277900000002 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840568+0000", 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.780667792000003, 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39904099900000001, 2024-09-15T21:01:26.274 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441605+0000", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441611+0000", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441652+0000", 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0890999999999999e-05 2024-09-15T21:01:26.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441671+0000", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9059000000000002e-05 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840619+0000", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39894794099999997 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840645+0000", 2024-09-15T21:01:26.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6064999999999999e-05 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:498 3.8s0 3:15433346:::smithi02732947-2:head [stat,delete] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.572445901999998, 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40308387200000001, 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 498 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649829+0000", 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0460000000000001e-06 2024-09-15T21:01:26.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649844+0000", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4397e-05 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649883+0000", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804999999999999e-05 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649901+0000", 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8145999999999999e-05 2024-09-15T21:01:26.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.730873+0000", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080971873 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.731287+0000", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000414008 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.733618+0000", 2024-09-15T21:01:26.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023318610000000002 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.735452+0000", 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018332019999999999 2024-09-15T21:01:26.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052787+0000", 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.31733539399999999 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052910+0000", 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012313999999999999 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.285 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:500 3.es0 3:7cf15e4b:::smithi02732947-48:head [copy-from ver 8] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.581356060000001, 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.72001296999999997, 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.286 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 500 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:26.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640919+0000", 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3100000000000001e-06 2024-09-15T21:01:26.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640934+0000", 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.491e-05 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640969+0000", 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4686e-05 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640986+0000", 2024-09-15T21:01:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7048999999999999e-05 2024-09-15T21:01:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641119+0000", 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013317 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641136+0000", 2024-09-15T21:01:26.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6558999999999998e-05 2024-09-15T21:01:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.845985+0000", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20484875999999999 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846249+0000", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026397900000000001 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.056421+0000", 2024-09-15T21:01:26.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210172042 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059407+0000", 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029868300000000002 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360789+0000", 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30138199900000001 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360929+0000", 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013967600000000001 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:501 3.es0 3:7cf15e4b:::smithi02732947-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.581245319000001, 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41837276400000001, 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:26.297 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 501 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641028+0000", 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641035+0000", 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641057+0000", 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1382e-05 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641079+0000", 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1733999999999999e-05 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741365+0000", 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10028688500000001 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741386+0000", 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0423000000000001e-05 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846689+0000", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105302942 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846826+0000", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137503 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.058847+0000", 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21202036799999999 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059116+0000", 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026980899999999998 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059400+0000", 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028316499999999999 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(osd.0.17:3 3.8s0 3:15433346:::smithi02732947-2:head [assert-version v76,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.581652717000001, 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51149464600000005, 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640622+0000", 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7099999999999999e-06 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640640+0000", 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7566000000000001e-05 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640661+0000", 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0971e-05 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640677+0000", 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6226e-05 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.740946+0000", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100268967 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.740973+0000", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6968e-05 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.041224+0000", 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30025154500000001 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.041385+0000", 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016017200000000001 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.152114+0000", 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110729521 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:832 3.8s0 3:15433346:::smithi02732947-2:head [append 0~720896] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5274134589999999, 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.57907397599999999, 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 832 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694862+0000", 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2059999999999999e-06 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696156+0000", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012936379999999999 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696190+0000", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4125000000000003e-05 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696208+0000", 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8442000000000001e-05 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696374+0000", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166133 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696390+0000", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5885000000000001e-05 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749483+0000", 2024-09-15T21:01:26.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053093057999999999 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749694+0000", 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021056300000000001 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.752168+0000", 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024746529999999998 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763652+0000", 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011483446 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273791+0000", 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51013918300000005 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273933+0000", 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014164399999999999 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:833 3.8s0 3:15433346:::smithi02732947-2:head [append 0~622592] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.522315098, 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67449799700000002, 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.315 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 833 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699960+0000", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.88e-06 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701223+0000", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012632870000000001 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701257+0000", 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4004999999999997e-05 2024-09-15T21:01:26.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701280+0000", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3144999999999999e-05 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.754820+0000", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053539760999999998 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.755101+0000", 2024-09-15T21:01:26.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028043799999999999 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.758090+0000", 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002989317 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763734+0000", 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056437930000000002 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374313+0000", 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61057963000000004 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374455+0000", 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141741 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:834 3.8s0 3:15433346:::smithi02732947-2:head [append 0~688128] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:26.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5163737509999997, 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66884350100000001, 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 834 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:26.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705901+0000", 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707380+0000", 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014797160000000001 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707409+0000", 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8951000000000001e-05 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707431+0000", 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1412000000000001e-05 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759507+0000", 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052076560000000001 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759651+0000", 2024-09-15T21:01:26.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014353099999999999 2024-09-15T21:01:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.762259+0000", 2024-09-15T21:01:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002607786 2024-09-15T21:01:26.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.864069+0000", 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101810885 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374639+0000", 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51056974700000002 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374742+0000", 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102773 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:835 3.8s0 3:15433346:::smithi02732947-2:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5141754589999996, 2024-09-15T21:01:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.667032505, 2024-09-15T21:01:26.330 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 835 2024-09-15T21:01:26.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708099+0000", 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.553e-06 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708461+0000", 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036148899999999999 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708494+0000", 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3674999999999999e-05 2024-09-15T21:01:26.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708517+0000", 2024-09-15T21:01:26.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2084999999999999e-05 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967220+0000", 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25870379199999999 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967394+0000", 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017396100000000001 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.968720+0000", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325737 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077387+0000", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10866701600000001 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374989+0000", 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29760241199999998 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375129+0000", 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013978499999999999 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:836 3.8s0 3:15433346:::smithi02732947-2:head [setxattr _header (56) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5137178899999997, 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66703066, 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 836 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708556+0000", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8670000000000001e-06 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708565+0000", 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5269999999999998e-06 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708590+0000", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5001e-05 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708606+0000", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6776e-05 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.969953+0000", 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26134606799999999 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.970101+0000", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014809999999999999 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971049+0000", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000948678 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077391+0000", 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10634163100000001 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375445+0000", 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298053911 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375585+0000", 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014010100000000001 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:837 3.8s0 3:15433346:::smithi02732947-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.344 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.5136391250000001, 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87430144200000004, 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 837 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708634+0000", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.364e-06 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708641+0000", 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9229999999999996e-06 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708665+0000", 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3515000000000001e-05 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708681+0000", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6395000000000001e-05 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971593+0000", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26291159200000003 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971759+0000", 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165966 2024-09-15T21:01:26.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378011+0000", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.406252055 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378267+0000", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025595099999999998 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.582935+0000", 2024-09-15T21:01:26.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20466768099999999 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:844 3.0s0 3:0196c8aa:::smithi02732947-4:head [create,setxattr lnO\\b_Vrmg\\YoalgS_ap]_oaNZQ[WWtVp\\Pqajab (15856) in=40b,setxattr lk^eORZ`gVcm``\\pr^dth`]NqmVUQkUObOacmWfh (699) in=40b,setxattr alfebafpqnNmSoe_jrs[jlogjdclQl]SfW^_opiR (911) in=40b,setxattr ]tVYcbfsnki]TcRbjSXlkPP\\URmibcZ (559) in=31b,setxattr cS_QsbNZRaNftQNafpmOXrSq[QTT^a`bjr^RjcXe (19716) in=40b,setxattr lWQarRah[_Ri`aaaYU\\YttQj[iZtlmkV]_giaXU] (1119) in=40b,setxattr afP^ZXR]oT[nYcXcXtmS^YhRRO`ah[[igtU\\maam (3576) in=40b,setxattr lfUpnr]dS`jeYdSeReTo^[[scdZRhaXsnpNagRV] (417) in=40b,setxattr oRZflPsTY\\iqWn_qits]aghfipZmOmndmY\\_fdsm (453) in=40b,setxattr cPXrXtr[heq_Y[rbnflZtdiTN[ejbhn]QUcpclY[ (10312) in=40b,setxattr t_S^QjNpSe[mtQWcjpeoQbsQWZqNlnhb`jXVNbkc (1647) in=40b,setxattr QhqWq\\srNqhaWrNUTpj\\lra]SNOZnScUZTX`NXWh (1717) in=40b,setxattr [VXbj[Sc_ttR`ab\\tdcTOlesaQOosdTTQnNtqnS\\ (1329) in=40b,setxattr kOPlXRXq\\neZk[`Qch\\fXP`]t`NWhTlj_XPZhlSY (4560) in=40b,setxattr P^eT`anhlNcnlqOfQsV^QOeZhkX^VbfPm`aNNkSN (1401) in=40b,setxattr gTqoYWiXq`b\\`jd\\V\\URmtOrZernYT_aSTkmR_lV (1715) in=40b,setxattr immbP]\\[qVq`fc\\]bYS\\lmgX[Sk_nfrSS_pTheVl (17540) in=40b,setxattr dRRpo\\[dNWhVPhqhmg[pqo_][oUNQgkOpkpaRPhS (1017) in=40b,setxattr [\\\\ofp^gsRlqNh]W^sW`[[qhU_rb\\qdg\\cffde_g (13964) in=40b,setxattr \\d[kjqp`[PR] (559) in=12b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4818974010000003, 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.63608215899999998, 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:26.350 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 844 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:26.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740405+0000", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0065000000000001e-05 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740634+0000", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229232 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740675+0000", 2024-09-15T21:01:26.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0556999999999998e-05 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740697+0000", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2138999999999999e-05 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971827+0000", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23113030700000001 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.972183+0000", 2024-09-15T21:01:26.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035558600000000002 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974826+0000", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026435999999999999 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077486+0000", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102659311 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375591+0000", 2024-09-15T21:01:26.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298105283 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.376457+0000", 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086607899999999996 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:26.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:26.357 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:26.350+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:26.357 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:26.457 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:26.451+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:26.496 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:26.496 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:26.496 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:26.497 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:26.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:26.559 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:26.555+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:26.565 INFO:tasks.rados.rados.0.smithi027.stdout:347: done (4 left) 2024-09-15T21:01:26.565 INFO:tasks.rados.rados.0.smithi027.stdout:348: copy_from oid 34 from oid 38 current snap is 31 2024-09-15T21:01:26.565 INFO:tasks.rados.rados.0.smithi027.stdout:330: finishing rollback tid 1 to smithi02732947-8 2024-09-15T21:01:26.565 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 213 (ObjNum 108 snap 19 seq_num 108) dirty exists 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:330: finishing rollback tid 2 to smithi02732947-8 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 79 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:345: finishing rollback tid 0 to smithi02732947-6 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:345: finishing rollback tid 1 to smithi02732947-6 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:345: finishing rollback tid 2 to smithi02732947-6 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 6 v 63 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:330: done (4 left) 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:345: done (3 left) 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:346: done (2 left) 2024-09-15T21:01:26.566 INFO:tasks.rados.rados.0.smithi027.stdout:349: snap_create 2024-09-15T21:01:26.634 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:26.634 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:26.634 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:26.635 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:26.635 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:26.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:26.660 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:26.658+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:26.680 INFO:tasks.rados.rados.0.smithi027.stdout:348: finishing copy_from to smithi02732947-34 2024-09-15T21:01:26.680 INFO:tasks.rados.rados.0.smithi027.stdout:348: got expected ENOENT (src dne) 2024-09-15T21:01:26.681 INFO:tasks.rados.rados.0.smithi027.stdout:348: finishing copy_from racing read to smithi02732947-34 2024-09-15T21:01:26.761 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:26.759+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:26.783 INFO:tasks.rados.rados.0.smithi027.stdout:339: finishing rollback tid 1 to smithi02732947-43 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.543453552000003, 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:01:26.814 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:01:26.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.543155413000001, 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:01:26.820 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:01:26.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:01:26.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.543131119000002, 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:26.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:26.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:01:26.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.336068656999998, 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:01:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:01:26.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:01:26.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.682832613999999, 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:01:26.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:01:26.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:26.840 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.582667555, 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:26.841 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:01:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:01:26.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 55.582622884999999, 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.846 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:01:26.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:01:26.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.665151285999997, 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:01:26.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.665059898999999, 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:01:26.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:01:26.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:359 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.864 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.50170851, 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35050568300000001, 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 359 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306330+0000", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306332+0000", 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8339999999999999e-06 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306628+0000", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029618500000000002 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306645+0000", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6518e-05 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306653+0000", 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1960000000000008e-06 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306721+0000", 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7945000000000001e-05 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306833+0000", 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112538 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.308213+0000", 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001379915 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.309315+0000", 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001102107 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656634+0000", 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34731901900000001 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.656836+0000", 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020142599999999999 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:26.870 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.501362739000001, 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:01:26.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:01:26.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:01:26.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:361 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~262144] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.400917708999998, 2024-09-15T21:01:26.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35217888200000003, 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 361 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407121+0000", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407122+0000", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.567e-06 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407417+0000", 2024-09-15T21:01:26.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029430100000000001 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407438+0000", 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1401000000000001e-05 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407450+0000", 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2036e-05 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409764+0000", 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0023141680000000001 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.409913+0000", 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000148777 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.411230+0000", 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00131726 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413635+0000", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024047719999999999 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759187+0000", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34555227100000002 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759300+0000", 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000112329 2024-09-15T21:01:26.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:362 3.6s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (56) in=7b] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.400566771999998, 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35221260100000001, 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 362 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407473+0000", 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407479+0000", 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5899999999999996e-06 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407492+0000", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2435999999999999e-05 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407501+0000", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5589999999999994e-06 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412116+0000", 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046143970000000001 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412250+0000", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013454100000000001 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413018+0000", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076778700000000003 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413954+0000", 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000935691 2024-09-15T21:01:26.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759576+0000", 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34562273999999998 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759684+0000", 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010788 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.400505975999998, 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:01:26.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:01:26.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:426 3.6s0 3:6d8f48d3:::smithi02732947-20:head [copy-from ver 70] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.552581365000002, 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32441989700000001, 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 426 2024-09-15T21:01:26.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255457+0000", 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255461+0000", 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255475+0000", 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.419e-05 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255507+0000", 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1507999999999997e-05 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255522+0000", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5056e-05 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255654+0000", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013243 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.255940+0000", 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028538199999999999 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.470481+0000", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21454129299999999 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.575970+0000", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105488665 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579610+0000", 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003640636 2024-09-15T21:01:26.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.579877+0000", 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000266894 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:675 3.6s0 3:6d8f48d3:::smithi02732947-20:head [read 618203~905509,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.9038255, 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.323741204, 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 675 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.899 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904213+0000", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904214+0000", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6000000000000003e-07 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904219+0000", 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0969999999999996e-06 2024-09-15T21:01:26.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904231+0000", 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1613e-05 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904248+0000", 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7232999999999999e-05 2024-09-15T21:01:26.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017097+0000", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.11284968300000001 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.017255+0000", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015797900000000001 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.227954+0000", 2024-09-15T21:01:26.902 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.210698839 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:772 3.1s0 3:81f89603:::smithi02732947-29:head [copy-from ver 76] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.173373022, 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.64200396500000001, 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.903 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:26.904 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634669+0000", 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4699999999999998e-06 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634686+0000", 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7204999999999999e-05 2024-09-15T21:01:26.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634729+0000", 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3251999999999999e-05 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634756+0000", 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6271e-05 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634873+0000", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116989 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634882+0000", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638177+0000", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003295035 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.907 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638307+0000", 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013013599999999999 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.165965+0000", 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52765778900000004 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.171723+0000", 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057583169999999998 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.908 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276516+0000", 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104793367 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276669+0000", 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000153046 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.909 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:773 3.1s0 3:81f89603:::smithi02732947-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 11.173229586, 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.53435976299999999, 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 773 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.910 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634811+0000", 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.804e-06 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.911 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634818+0000", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4549999999999998e-06 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634843+0000", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4895000000000001e-05 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634866+0000", 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3198e-05 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634983+0000", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000117088 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634995+0000", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1912e-05 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638554+0000", 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00355902 2024-09-15T21:01:26.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638636+0000", 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1588999999999994e-05 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169005+0000", 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.530369117 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.914 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169073+0000", 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7940000000000003e-05 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169169+0000", 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5744999999999999e-05 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:850 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 1e] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:26.915 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.1424006240000004, 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3063522750000001, 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 850 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:26.916 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665639+0000", 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8500000000000001e-07 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.917 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665642+0000", 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999999e-06 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665651+0000", 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4620000000000002e-06 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765755+0000", 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100104183 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.918 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766158+0000", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040273000000000002 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766308+0000", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015033700000000001 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766461+0000", 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015210000000000001 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371273+0000", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6048122309999999 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371725+0000", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00045230200000000002 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663049+0000", 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29132430799999998 2024-09-15T21:01:26.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663312+0000", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026254099999999998 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.665305+0000", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019926309999999999 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870050+0000", 2024-09-15T21:01:26.921 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20474537700000001 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971470+0000", 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10142055 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971990+0000", 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00051970900000000003 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:851 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.0422404759999999, 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3075131120000001, 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 851 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:26.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765800+0000", 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1660000000000001e-06 2024-09-15T21:01:26.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765811+0000", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0787e-05 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765844+0000", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2492e-05 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765866+0000", 2024-09-15T21:01:26.925 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2135999999999999e-05 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769059+0000", 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031937459999999999 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769218+0000", 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015836100000000001 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.926 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371923+0000", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6027050859999998 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.372518+0000", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059501300000000003 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666553+0000", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29403548499999999 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:26.927 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666683+0000", 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129827 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.667532+0000", 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084906999999999995 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870061+0000", 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20252906200000001 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.072834+0000", 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202773013 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.073311+0000", 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047686799999999998 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:26.929 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:26.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:26.931 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:26.927+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:26.978 INFO:tasks.rados.rados.0.smithi027.stdout:339: finishing rollback tid 2 to smithi02732947-43 2024-09-15T21:01:26.978 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 43 v 17 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T21:01:27.032 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:27.027+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDPGPush(3.9s5 93/84 [PushOp(3:9b48fc7f:::smithi02732947-22:1f, version: 90'70, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:9b48fc7f:::smithi02732947-22:1f@90'70, size: 3735552, copy_subset: [], clone_subset: {}, snapset: 1f=[]:{1f: [1f, 1e, 1d, 1b, 1a, 19, 18, 16]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:26.991284+0000", 2024-09-15T21:01:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.071505479999999996, 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.071626238999999994, 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991284+0000", 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991284+0000", 2024-09-15T21:01:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991285+0000", 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2099999999999996e-07 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991614+0000", 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032873100000000002 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991635+0000", 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1166e-05 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.991653+0000", 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7479999999999999e-05 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T21:01:27.067 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:27.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:27.133 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:27.127+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:27.234 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:27.230+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:27.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:27.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:27.331+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:27.383 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.383 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:27.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:27.383 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:27.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.622514021000001, 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:01:27.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:01:27.387 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.622466963000001, 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:27.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:01:27.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.622439163000003, 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:01:27.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.622413768999998, 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.396 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:27.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:01:27.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.317034546999999, 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:01:27.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.405 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.316473586999997, 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:01:27.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:01:27.409 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:01:27.410 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.216315203999997, 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.411 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:01:27.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:01:27.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:01:27.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:01:27.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.216218734000002, 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:27.417 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:01:27.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.422 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.215261304000002, 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:01:27.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:01:27.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.215191797999999, 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:27.428 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:01:27.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:27.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.115424093000001, 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.434 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.435 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.437 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 63.011751920000002, 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:01:27.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.998492149, 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:01:27.441 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.996551722, 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:01:27.445 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:01:27.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.806898095000001, 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.449 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.452 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.397887666000003, 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.456 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.390366258, 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.197486648000002, 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:01:27.460 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:01:27.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.196483569999998, 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.464 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:01:27.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:27.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.531930764999998, 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:01:27.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:01:27.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.471 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:27.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:27.473 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:27.470+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:27.568 INFO:tasks.rados.rados.0.smithi027.stdout:339: done (2 left) 2024-09-15T21:01:27.568 INFO:tasks.rados.rados.0.smithi027.stdout:348: done (1 left) 2024-09-15T21:01:27.568 INFO:tasks.rados.rados.0.smithi027.stdout:349: done (0 left) 2024-09-15T21:01:27.568 INFO:tasks.rados.rados.0.smithi027.stdout:350: copy_from oid 27 from oid 39 current snap is 32 2024-09-15T21:01:27.569 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 8 current snap is 32 2024-09-15T21:01:27.569 INFO:tasks.rados.rados.0.smithi027.stdout:351: seq_num 141 ranges {2129920=425984,2555904=573440,3129344=425984,3555328=524288,4079616=425984} 2024-09-15T21:01:27.574 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:27.566+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:27.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:01:27.599 INFO:tasks.rados.rados.0.smithi027.stdout:351: writing smithi02732947-8 from 2129920 to 2555904 tid 1 2024-09-15T21:01:27.605 INFO:tasks.rados.rados.0.smithi027.stdout:351: writing smithi02732947-8 from 2555904 to 3129344 tid 2 2024-09-15T21:01:27.610 INFO:tasks.rados.rados.0.smithi027.stdout:351: writing smithi02732947-8 from 3129344 to 3555328 tid 3 2024-09-15T21:01:27.611 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:27.611 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:27.611 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:27.616 INFO:tasks.rados.rados.0.smithi027.stdout:351: writing smithi02732947-8 from 3555328 to 4079616 tid 4 2024-09-15T21:01:27.621 INFO:tasks.rados.rados.0.smithi027.stdout:351: writing smithi02732947-8 from 4079616 to 4505600 tid 5 2024-09-15T21:01:27.621 INFO:tasks.rados.rados.0.smithi027.stdout:352: copy_from oid 45 from oid 50 current snap is 32 2024-09-15T21:01:27.621 INFO:tasks.rados.rados.0.smithi027.stdout:353: snap_create 2024-09-15T21:01:27.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:27.675 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:27.671+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:27.694 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:27.695 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:27.695 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-15T21:01:27.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:01:27.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:27.715+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,4,5 2024-09-15T21:01:27.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:27.719+0000 7fde7490f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:27.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:27.719+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4 to read 0,1,4,5 2024-09-15T21:01:27.768 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:27.763+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:27.768 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:27.763+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:27.769 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:27.763+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:01:27.775 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:27.770+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:27.803 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.803 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:27.803 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:27.803 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:27.803 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:27.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:27.876 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:27.871+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:27.889 INFO:tasks.rados.rados.0.smithi027.stdout:350: finishing copy_from racing read to smithi02732947-27 2024-09-15T21:01:27.916 INFO:tasks.rados.rados.0.smithi027.stdout:352: finishing copy_from racing read to smithi02732947-45 2024-09-15T21:01:27.977 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:27.975+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 62.395470918000001, 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:01:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:01:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 58.907518529000001, 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.204504899, 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:01:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 48/28 ECSubRead(tid=132, to_read={3:a5da2594:::smithi02732947-25:head=0,315392,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.527987414999998, 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20088298299999999, 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463043+0000", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463058+0000", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4732999999999999e-05 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.563142+0000", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008391699999999 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663346+0000", 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100204423 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663918+0000", 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057127899999999995 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.119963859000002, 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30162102800000001, 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871059+0000", 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871100+0000", 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0321999999999998e-05 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871128+0000", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8208000000000002e-05 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871152+0000", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4246000000000001e-05 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971464+0000", 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312292 2024-09-15T21:01:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971482+0000", 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7176e-05 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172613+0000", 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20113102099999999 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172679+0000", 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6762999999999998e-05 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.119658999999999, 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20089554500000001, 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871364+0000", 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871378+0000", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3706000000000001e-05 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871408+0000", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0574999999999998e-05 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971547+0000", 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013905400000001 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072019+0000", 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100471385 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072259+0000", 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023990200000000001 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.092025487999997, 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20140428699999999, 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898998+0000", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4980000000000001e-06 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899107+0000", 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010874699999999999 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899130+0000", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.268e-05 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999226+0000", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096764 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999366+0000", 2024-09-15T21:01:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139231 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999377+0000", 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1097000000000001e-05 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100334+0000", 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100957065 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100401+0000", 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7205000000000005e-05 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=180, reqid=client.4641.0:451, at_version=60'74, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 36.090246929000003, 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.200850169, 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900776+0000", 2024-09-15T21:01:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900867+0000", 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1712000000000006e-05 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900880+0000", 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.218e-05 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001054+0000", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10017451600000001 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001292+0000", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023806500000000001 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001309+0000", 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101581+0000", 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027267099999999 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101625+0000", 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3965999999999999e-05 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 68/53 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=68'11, trim_to=0'0, roll_forward_to=67'9))", 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 28.243246683999999, 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20168053699999999, 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747776+0000", 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8e-07 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747806+0000", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0182e-05 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747817+0000", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0985000000000001e-05 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.847897+0000", 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100080024 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948088+0000", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100191302 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948106+0000", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7869000000000001e-05 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949395+0000", 2024-09-15T21:01:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001288909 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949456+0000", 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1086000000000004e-05 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=236, reqid=osd.5.0:236, at_version=69'114, trim_to=0'0, roll_forward_to=68'112))", 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.939649810999999, 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.201432585, 2024-09-15T21:01:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051375+0000", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8629999999999999e-06 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051452+0000", 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6847000000000005e-05 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051489+0000", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7295999999999997e-05 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151633+0000", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100143989 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151772+0000", 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138505 2024-09-15T21:01:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151792+0000", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0516000000000001e-05 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252713+0000", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10092024300000001 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252805+0000", 2024-09-15T21:01:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2325999999999997e-05 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=238, reqid=osd.5.0:237, at_version=0'0, trim_to=0'0, roll_forward_to=69'116))", 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.532546258, 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202021808, 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458478+0000", 2024-09-15T21:01:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2929999999999999e-06 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458507+0000", 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9183000000000002e-05 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458535+0000", 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7512999999999999e-05 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.558646+0000", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100111397 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559059+0000", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041233799999999999 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559089+0000", 2024-09-15T21:01:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0145999999999999e-05 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660438+0000", 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134961000000001 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660498+0000", 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9327999999999998e-05 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=56, reqid=client.4641.0:576, at_version=69'122, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.280095904, 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30235248100000001, 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710927+0000", 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7000000000000001e-07 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711169+0000", 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242449 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711184+0000", 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4956999999999999e-05 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811338+0000", 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015429200000001 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911614+0000", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027530699999999 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911632+0000", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7963000000000001e-05 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.012893+0000", 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101261014 2024-09-15T21:01:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013279+0000", 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038622899999999998 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=57, reqid=client.4641.0:577, at_version=69'123, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 26.179629082000002, 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20197553200000001, 2024-09-15T21:01:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811394+0000", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1400000000000001e-06 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811565+0000", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017046300000000001 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811589+0000", 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4229000000000001e-05 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811601+0000", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2194e-05 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912520+0000", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100918439 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912530+0000", 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0720000000000001e-05 2024-09-15T21:01:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013321+0000", 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100790989 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013369+0000", 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7358000000000003e-05 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 75/59 ECSubWrite(tid=266, reqid=osd.5.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 21.223673861000002, 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30105612199999998, 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767349+0000", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.133e-06 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767364+0000", 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4146e-05 2024-09-15T21:01:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767381+0000", 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7144000000000001e-05 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.867460+0000", 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100079691 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967708+0000", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100247187 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967719+0000", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0937e-05 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068375+0000", 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100655967 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068404+0000", 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9916999999999998e-05 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 78/59 ECSubWrite(tid=268, reqid=client.4641.0:643, at_version=78'141, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.000695213, 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.203284137, 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990328+0000", 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990551+0000", 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022275300000000001 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990576+0000", 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5414000000000001e-05 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.090713+0000", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013612099999999 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.190988+0000", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027497 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.191018+0000", 2024-09-15T21:01:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0602000000000003e-05 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193517+0000", 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024985929999999999 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193611+0000", 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4324e-05 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 81/57 ECSubWrite(tid=93, reqid=client.4641.0:730, at_version=81'151, trim_to=0'0, roll_forward_to=81'150))", 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 12.565138865, 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.109107326, 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425883+0000", 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.425886+0000", 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6400000000000001e-06 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431096+0000", 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0052103649999999998 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431129+0000", 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2317000000000002e-05 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.431145+0000", 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6651999999999999e-05 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.531373+0000", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100227812 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.531390+0000", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6818000000000001e-05 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.534901+0000", 2024-09-15T21:01:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0035111080000000002 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.534991+0000", 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.9614e-05 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=130, reqid=client.4641.0:832, at_version=87'14, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 7.2397379759999998, 2024-09-15T21:01:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.188249429, 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751285+0000", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751828+0000", 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054304400000000001 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751839+0000", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751846+0000", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1110000000000002e-06 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836633+0000", 2024-09-15T21:01:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084786880999999995 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836646+0000", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2927e-05 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939450+0000", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10280324 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939534+0000", 2024-09-15T21:01:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4116000000000006e-05 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=131, reqid=client.4641.0:833, at_version=87'15, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 7.2340949190000003, 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.18273326100000001, 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756928+0000", 2024-09-15T21:01:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757088+0000", 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015978499999999999 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757098+0000", 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0482999999999999e-05 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757103+0000", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2259999999999996e-06 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937980+0000", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.180876436 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937987+0000", 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5440000000000001e-06 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939584+0000", 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001597137 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939661+0000", 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6057999999999996e-05 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=132, reqid=client.4641.0:834, at_version=87'16, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 7.2296211929999998, 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17882461699999999, 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761402+0000", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2500000000000005e-07 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761528+0000", 2024-09-15T21:01:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012658 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761541+0000", 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2294e-05 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761547+0000", 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.117e-06 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938455+0000", 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.176908119 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938472+0000", 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6898999999999999e-05 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940190+0000", 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001718687 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940226+0000", 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5196e-05 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 88/57 ECSubWrite(tid=112, reqid=client.4641.0:846, at_version=88'165, trim_to=0'0, roll_forward_to=87'163))", 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.120385+0000", 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.8706375189999997, 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.110639336, 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:28.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.120385+0000", 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.120385+0000", 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.120385+0000", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.126614+0000", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0062290239999999997 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.126632+0000", 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7978e-05 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.126641+0000", 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2450000000000004e-06 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.226812+0000", 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100170641 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.226825+0000", 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3227000000000001e-05 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.230972+0000", 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0041468709999999999 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.231024+0000", 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.215e-05 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:28.086 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:28.088 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:28.089 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:01:28.090 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-15T21:01:27.565675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":162595379,"num_objects":96,"num_object_clones":45,"num_object_copies":744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":5,"num_read":448,"num_read_kb":147318,"num_write":908,"num_write_kb":185192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":93552678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1078,"ondisk_log_size":1078,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":715836,"kb_used_data":333188,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1131746244,"statfs":{"total":1159641169920,"available":1158908153856,"internally_reserved":0,"allocated":341184512,"data_stored":480655411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4669440,"num_objects":4,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":-1,"num_read":18,"num_read_kb":30236,"num_write":33,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18104320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021722"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":295,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.669752+0000","last_change":"2024-09-15T21:01:26.670669+0000","last_active":"2024-09-15T21:01:26.669752+0000","last_peered":"2024-09-15T21:01:26.669752+0000","last_clean":"2024-09-15T21:01:26.669752+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:01:26.669752+0000","last_undegraded":"2024-09-15T21:01:26.669752+0000","last_fullsized":"2024-09-15T21:01:26.669752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:43.739509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015720420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.e","version":"81'20","reported_seq":319,"reported_epoch":89,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:22.669383+0000","last_change":"2024-09-15T21:01:19.052837+0000","last_active":"2024-09-15T21:01:22.669383+0000","last_peered":"2024-09-15T21:01:22.669383+0000","last_clean":"2024-09-15T21:01:15.841647+0000","last_became_active":"2024-09-15T21:01:18.950877+0000","last_became_peered":"2024-09-15T21:01:18.950877+0000","last_unstale":"2024-09-15T21:01:22.669383+0000","last_undegraded":"2024-09-15T21:01:18.549467+0000","last_fullsized":"2024-09-15T21:01:18.548517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30587246800000001,"stat_sum":{"num_bytes":786432,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":10,"num_read_kb":1257,"num_write":8,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,2147483647,4,0,9,3,10],"avail_no_missing":["11(0)","0(4)","3(6)","4(3)","6(2)","8(1)","9(5)","10(7)"],"object_location_counts":[{"shards":"0(4),3(6),4(3),6(2),8(1),9(5),10(7),11(0)","objects":2}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"3.d","version":"91'27","reported_seq":333,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.669561+0000","last_change":"2024-09-15T21:01:26.670252+0000","last_active":"2024-09-15T21:01:26.669561+0000","last_peered":"2024-09-15T21:01:26.669561+0000","last_clean":"2024-09-15T21:01:26.669561+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:01:26.669561+0000","last_undegraded":"2024-09-15T21:01:26.669561+0000","last_fullsized":"2024-09-15T21:01:26.669561+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:31.505728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012280990000000001,"stat_sum":{"num_bytes":1687552,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":4922,"num_write":30,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.c","version":"91'87","reported_seq":384,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.673295+0000","last_change":"2024-09-15T21:01:26.674171+0000","last_active":"2024-09-15T21:01:26.673295+0000","last_peered":"2024-09-15T21:01:26.673295+0000","last_clean":"2024-09-15T21:01:26.673295+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:01:26.673295+0000","last_undegraded":"2024-09-15T21:01:26.673295+0000","last_fullsized":"2024-09-15T21:01:26.673295+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016471140000000001,"stat_sum":{"num_bytes":10272768,"num_objects":6,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":11640,"num_write":53,"num_write_kb":10576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.b","version":"91'98","reported_seq":437,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.659535+0000","last_change":"2024-09-15T21:01:24.761785+0000","last_active":"2024-09-15T21:01:25.659535+0000","last_peered":"2024-09-15T21:01:25.659535+0000","last_clean":"2024-09-15T21:01:25.659535+0000","last_became_active":"2024-09-15T21:00:51.891415+0000","last_became_peered":"2024-09-15T21:00:51.891415+0000","last_unstale":"2024-09-15T21:01:25.659535+0000","last_undegraded":"2024-09-15T21:01:25.659535+0000","last_fullsized":"2024-09-15T21:01:25.659535+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21285827199999999,"stat_sum":{"num_bytes":12288000,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":29767,"num_write":48,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"3.a","version":"81'15","reported_seq":333,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.573322+0000","last_change":"2024-09-15T21:01:26.573322+0000","last_active":"2024-09-15T21:01:26.573322+0000","last_peered":"2024-09-15T21:01:26.573322+0000","last_clean":"2024-09-15T21:01:26.573322+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:01:26.573322+0000","last_undegraded":"2024-09-15T21:01:26.573322+0000","last_fullsized":"2024-09-15T21:01:26.573322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:38.180718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036363350000000001,"stat_sum":{"num_bytes":4423680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":12576,"num_write":12,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.9","version":"90'75","reported_seq":431,"reported_epoch":92,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:26.445968+0000","last_change":"2024-09-15T21:01:25.287121+0000","last_active":"2024-09-15T21:01:26.445968+0000","last_peered":"2024-09-15T21:01:26.445968+0000","last_clean":"2024-09-15T21:01:15.739666+0000","last_became_active":"2024-09-15T21:01:18.958099+0000","last_became_peered":"2024-09-15T21:01:18.958099+0000","last_unstale":"2024-09-15T21:01:26.445968+0000","last_undegraded":"2024-09-15T21:01:18.547002+0000","last_fullsized":"2024-09-15T21:01:18.546025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.11977742199999999,"stat_sum":{"num_bytes":11337728,"num_objects":6,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":3263,"num_write":78,"num_write_kb":13648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":12599296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,2147483647,4,10],"avail_no_missing":["8(0)","4(6)","5(4)","6(5)","7(2)","9(3)","10(7)","11(1)"],"object_location_counts":[{"shards":"4(6),5(4),6(5),7(2),8(0),9(3),10(7),11(1)","objects":1},{"shards":"4(6),5(4),7(2),8(0),9(3),10(7),11(1)","objects":3}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"3.8","version":"87'18","reported_seq":313,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-09-15T21:01:22.769372+0000","last_change":"2024-09-15T21:01:22.769474+0000","last_active":"2024-09-15T21:01:22.769372+0000","last_peered":"2024-09-15T21:01:22.769372+0000","last_clean":"2024-09-15T21:01:22.769372+0000","last_became_active":"2024-09-15T21:00:45.947614+0000","last_became_peered":"2024-09-15T21:00:45.947614+0000","last_unstale":"2024-09-15T21:01:22.769372+0000","last_undegraded":"2024-09-15T21:01:22.769372+0000","last_fullsized":"2024-09-15T21:01:22.769372+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20750359900000001,"stat_sum":{"num_bytes":5537792,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":10,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":180224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"1.0","version":"21'92","reported_seq":279,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.570841+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:01:26.570841+0000","last_peered":"2024-09-15T21:01:26.570841+0000","last_clean":"2024-09-15T21:01:26.570841+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:01:26.570841+0000","last_undegraded":"2024-09-15T21:01:26.570841+0000","last_fullsized":"2024-09-15T21:01:26.570841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.571813+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:01:26.571813+0000","last_peered":"2024-09-15T21:01:26.571813+0000","last_clean":"2024-09-15T21:01:26.571813+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:01:26.571813+0000","last_undegraded":"2024-09-15T21:01:26.571813+0000","last_fullsized":"2024-09-15T21:01:26.571813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"91'81","reported_seq":482,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.659454+0000","last_change":"2024-09-15T21:01:25.577106+0000","last_active":"2024-09-15T21:01:25.659454+0000","last_peered":"2024-09-15T21:01:25.659454+0000","last_clean":"2024-09-15T21:01:25.659454+0000","last_became_active":"2024-09-15T21:01:24.854584+0000","last_became_peered":"2024-09-15T21:01:24.854584+0000","last_unstale":"2024-09-15T21:01:25.659454+0000","last_undegraded":"2024-09-15T21:01:25.659454+0000","last_fullsized":"2024-09-15T21:01:25.659454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:40.125552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.721033272,"stat_sum":{"num_bytes":16908288,"num_objects":11,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":36,"num_read_kb":16548,"num_write":62,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":26951680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.570379+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:01:26.570379+0000","last_peered":"2024-09-15T21:01:26.570379+0000","last_clean":"2024-09-15T21:01:26.570379+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:01:26.570379+0000","last_undegraded":"2024-09-15T21:01:26.570379+0000","last_fullsized":"2024-09-15T21:01:26.570379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'44","reported_seq":385,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-15T21:01:25.971474+0000","last_change":"2024-09-15T21:01:25.868698+0000","last_active":"2024-09-15T21:01:25.971474+0000","last_peered":"2024-09-15T21:01:25.971474+0000","last_clean":"2024-09-15T21:01:16.276520+0000","last_became_active":"2024-09-15T21:01:18.868498+0000","last_became_peered":"2024-09-15T21:01:18.868498+0000","last_unstale":"2024-09-15T21:01:25.971474+0000","last_undegraded":"2024-09-15T21:01:25.971474+0000","last_fullsized":"2024-09-15T21:01:18.552642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20939993800000001,"stat_sum":{"num_bytes":5865472,"num_objects":5,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":17588,"num_write":41,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":13811712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,2147483647],"avail_no_missing":["0(0)","1(7)","4(1)","5(4)","6(6)","7(7)","9(5)","10(2)","11(3)"],"object_location_counts":[{"shards":"0(0),4(1),5(4),6(6),9(5),10(2),11(3)","objects":5}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.676739+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:01:26.676739+0000","last_peered":"2024-09-15T21:01:26.676739+0000","last_clean":"2024-09-15T21:01:26.676739+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:01:26.676739+0000","last_undegraded":"2024-09-15T21:01:26.676739+0000","last_fullsized":"2024-09-15T21:01:26.676739+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"87'40","reported_seq":358,"reported_epoch":88,"state":"peering","last_fresh":"2024-09-15T21:01:21.996511+0000","last_change":"2024-09-15T21:01:22.768427+0000","last_active":"2024-09-15T21:01:22.768427+0000","last_peered":"2024-09-15T21:01:21.996511+0000","last_clean":"2024-09-15T21:01:15.841090+0000","last_became_active":"2024-09-15T21:01:18.758248+0000","last_became_peered":"2024-09-15T21:01:18.758248+0000","last_unstale":"2024-09-15T21:01:21.996511+0000","last_undegraded":"2024-09-15T21:01:21.996511+0000","last_fullsized":"2024-09-15T21:01:18.547906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.607917771,"stat_sum":{"num_bytes":9306112,"num_objects":5,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":9578,"num_write":81,"num_write_kb":9088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13615104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,3,5,6,7,8,9],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":184,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.572026+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:01:26.572026+0000","last_peered":"2024-09-15T21:01:26.572026+0000","last_clean":"2024-09-15T21:01:26.572026+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:01:26.572026+0000","last_undegraded":"2024-09-15T21:01:26.572026+0000","last_fullsized":"2024-09-15T21:01:26.572026+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'46","reported_seq":338,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.556345+0000","last_change":"2024-09-15T21:01:25.067674+0000","last_active":"2024-09-15T21:01:25.556345+0000","last_peered":"2024-09-15T21:01:25.556345+0000","last_clean":"2024-09-15T21:01:25.556345+0000","last_became_active":"2024-09-15T21:00:20.410469+0000","last_became_peered":"2024-09-15T21:00:20.410469+0000","last_unstale":"2024-09-15T21:01:25.556345+0000","last_undegraded":"2024-09-15T21:01:25.556345+0000","last_fullsized":"2024-09-15T21:01:25.556345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:38.583272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50683778400000001,"stat_sum":{"num_bytes":12189696,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":16,"num_read_kb":2318,"num_write":38,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"3.5","version":"93'215","reported_seq":584,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.879135+0000","last_change":"2024-09-15T21:01:26.881464+0000","last_active":"2024-09-15T21:01:26.879135+0000","last_peered":"2024-09-15T21:01:26.879135+0000","last_clean":"2024-09-15T21:01:26.879135+0000","last_became_active":"2024-09-15T21:00:51.794785+0000","last_became_peered":"2024-09-15T21:00:51.794785+0000","last_unstale":"2024-09-15T21:01:26.879135+0000","last_undegraded":"2024-09-15T21:01:26.879135+0000","last_fullsized":"2024-09-15T21:01:26.879135+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:41.434597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20904110400000001,"stat_sum":{"num_bytes":32980992,"num_objects":17,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":6503,"num_write":172,"num_write_kb":38800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12353536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.559160+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:01:25.559160+0000","last_peered":"2024-09-15T21:01:25.559160+0000","last_clean":"2024-09-15T21:01:25.559160+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:01:25.559160+0000","last_undegraded":"2024-09-15T21:01:25.559160+0000","last_fullsized":"2024-09-15T21:01:25.559160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:10.241744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"89'169","reported_seq":461,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T21:01:23.068379+0000","last_change":"2024-09-15T21:01:23.070292+0000","last_active":"2024-09-15T21:01:23.068379+0000","last_peered":"2024-09-15T21:01:23.068379+0000","last_clean":"2024-09-15T21:01:23.068379+0000","last_became_active":"2024-09-15T21:00:50.048868+0000","last_became_peered":"2024-09-15T21:00:50.048868+0000","last_unstale":"2024-09-15T21:01:23.068379+0000","last_undegraded":"2024-09-15T21:01:23.068379+0000","last_fullsized":"2024-09-15T21:01:23.068379+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:47.100886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.308324351,"stat_sum":{"num_bytes":33603584,"num_objects":13,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":47,"num_read_kb":23998,"num_write":88,"num_write_kb":37232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[10,11,9,5,7,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"15","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.554253+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:01:25.554253+0000","last_peered":"2024-09-15T21:01:25.554253+0000","last_clean":"2024-09-15T21:01:25.554253+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:01:25.554253+0000","last_undegraded":"2024-09-15T21:01:25.554253+0000","last_fullsized":"2024-09-15T21:01:25.554253+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"90'7","reported_seq":318,"reported_epoch":92,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:26.381161+0000","last_change":"2024-09-15T21:01:26.274242+0000","last_active":"2024-09-15T21:01:26.381161+0000","last_peered":"2024-09-15T21:01:26.381161+0000","last_clean":"2024-09-15T21:01:15.826039+0000","last_became_active":"2024-09-15T21:01:19.058048+0000","last_became_peered":"2024-09-15T21:01:19.058048+0000","last_unstale":"2024-09-15T21:01:26.381161+0000","last_undegraded":"2024-09-15T21:01:18.551959+0000","last_fullsized":"2024-09-15T21:01:18.550012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063942700000000003,"stat_sum":{"num_bytes":1490944,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3466,"num_write":6,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2260992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,2147483647],"avail_no_missing":["3(0)","0(3)","4(6)","5(7)","7(2)","9(5)","10(4)","11(1)"],"object_location_counts":[{"shards":"0(3),3(0),4(6),7(2),9(5),10(4),11(1)","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"15","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.556370+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:01:25.556370+0000","last_peered":"2024-09-15T21:01:25.556370+0000","last_clean":"2024-09-15T21:01:25.556370+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:01:25.556370+0000","last_undegraded":"2024-09-15T21:01:25.556370+0000","last_fullsized":"2024-09-15T21:01:25.556370+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"91'41","reported_seq":370,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-15T21:01:25.559590+0000","last_change":"2024-09-15T21:01:25.061552+0000","last_active":"2024-09-15T21:01:25.559590+0000","last_peered":"2024-09-15T21:01:25.559590+0000","last_clean":"2024-09-15T21:01:25.559590+0000","last_became_active":"2024-09-15T21:00:48.749851+0000","last_became_peered":"2024-09-15T21:00:48.749851+0000","last_unstale":"2024-09-15T21:01:25.559590+0000","last_undegraded":"2024-09-15T21:01:25.559590+0000","last_fullsized":"2024-09-15T21:01:25.559590+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.408507752,"stat_sum":{"num_bytes":3325952,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":23,"num_read_kb":3828,"num_write":56,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2473984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"15","length":"1"},{"start":"19","length":"1"},{"start":"1c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-15T21:01:26.568150+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:01:26.568150+0000","last_peered":"2024-09-15T21:01:26.568150+0000","last_clean":"2024-09-15T21:01:26.568150+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:01:26.568150+0000","last_undegraded":"2024-09-15T21:01:26.568150+0000","last_fullsized":"2024-09-15T21:01:26.568150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:08.004258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":162004992,"num_objects":92,"num_object_clones":45,"num_object_copies":736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":5,"num_read":372,"num_read_kb":147254,"num_write":783,"num_write_kb":183040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":93552640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327925760,"data_stored":476004352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":983,"ondisk_log_size":983,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378646,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52368,"kb_used_data":21184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319472,"statfs":{"total":96636764160,"available":96583139328,"internally_reserved":0,"allocated":21692416,"data_stored":30877805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411349,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70100,"kb_used_data":34744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94301740,"statfs":{"total":96636764160,"available":96564981760,"internally_reserved":0,"allocated":35577856,"data_stored":48878460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411348,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64932,"kb_used_data":33736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306908,"statfs":{"total":96636764160,"available":96570273792,"internally_reserved":0,"allocated":34545664,"data_stored":46366509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411348,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56460,"kb_used_data":25276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315380,"statfs":{"total":96636764160,"available":96578949120,"internally_reserved":0,"allocated":25882624,"data_stored":37029565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411348,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63144,"kb_used_data":31960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308696,"statfs":{"total":96636764160,"available":96572104704,"internally_reserved":0,"allocated":32727040,"data_stored":47352461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444053,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62964,"kb_used_data":31768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308876,"statfs":{"total":96636764160,"available":96572289024,"internally_reserved":0,"allocated":32530432,"data_stored":46683837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444054,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69080,"kb_used_data":37884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302760,"statfs":{"total":96636764160,"available":96566026240,"internally_reserved":0,"allocated":38793216,"data_stored":55629501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444053,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73348,"kb_used_data":37996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94298492,"statfs":{"total":96636764160,"available":96561655808,"internally_reserved":0,"allocated":38907904,"data_stored":55756928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444052,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50752,"kb_used_data":19560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":20029440,"data_stored":28169475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444054,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":25120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":25722880,"data_stored":38565565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444052,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41400,"kb_used_data":10180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330440,"statfs":{"total":96636764160,"available":96594370560,"internally_reserved":0,"allocated":10424320,"data_stored":12014870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444053,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54972,"kb_used_data":23780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316868,"statfs":{"total":96636764160,"available":96580472832,"internally_reserved":0,"allocated":24350720,"data_stored":33330435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23281664,"data_stored":34164736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10117120,"data_stored":12648448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20598784,"data_stored":30195712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24641536,"data_stored":37892096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18960384,"data_stored":27504640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37810176,"data_stored":55074816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37711872,"data_stored":54792192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":31039488,"data_stored":46080000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24801280,"data_stored":38363136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":31449088,"data_stored":44793856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":33513472,"data_stored":46260224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":34000896,"data_stored":48234496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:01:28.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 26 2024-09-15T21:01:28.092 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:28.082+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:28.093 INFO:tasks.rados.rados.0.smithi027.stdout:350: finishing copy_from to smithi02732947-27 2024-09-15T21:01:28.093 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 27 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T21:01:28.103 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 1 to smithi02732947-8 2024-09-15T21:01:28.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:28.196 INFO:tasks.rados.rados.0.smithi027.stdout:352: finishing copy_from to smithi02732947-45 2024-09-15T21:01:28.196 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 45 v 155 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-15T21:01:28.196 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:28.191+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:28.217 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:28.217 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:28.217 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:28.217 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:28.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:28.290 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:28.282+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:28.386 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:28.386 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:28.386 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:28.391 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:28.386+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:28.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:28.402 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 2 to smithi02732947-8 2024-09-15T21:01:28.422 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:28.422 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:28.439 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:28.440 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:28.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:28.474 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:28.474 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:28.491 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:28.487+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:28.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:28.494 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:28.506 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 3 to smithi02732947-8 2024-09-15T21:01:28.515 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:28.515 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:28.517 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:28.517 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:28.528 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:28.540 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:28.541 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:01:28.543 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 3 pgp_num to 26 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:350: done (3 left) 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:352: done (2 left) 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:353: done (1 left) 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:354: read oid 32 snap -1 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:354: expect (ObjNum 96 snap 17 seq_num 96) 2024-09-15T21:01:28.547 INFO:tasks.rados.rados.0.smithi027.stdout:355: snap_remove snap 27 2024-09-15T21:01:28.593 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:28.586+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:28.655 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:28.671 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:28.671 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:28.685 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:28.685 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:28.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:28.694 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:28.686+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:28.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:28.712 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:28.739 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:28.739 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:28.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.751+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:28.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.751+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:28.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.751+0000 7f5af740c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:28.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.751+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:28.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.751+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:28.769 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:28.770 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:28.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:28.787+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:28.814 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:28.815 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:28.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:28.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.851+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:28.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.855+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:01:28.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.855+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:01:28.864 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.859+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:28.894 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:28.891+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:28.965 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.959+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:01:28.965 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:28.959+0000 7f5af3404640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,6 to read 0,1,4,5 2024-09-15T21:01:28.985 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:28.985 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:28.995 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:28.991+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:29.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:29.013 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:29.007+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:29.013 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:29.007+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:01:29.013 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:29.007+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,4,5 2024-09-15T21:01:29.096 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:29.090+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:29.169 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.169 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.184 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:29.197 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:29.191+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:29.298 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:29.290+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:29.305 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.305 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.319 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:29.399 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:29.391+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:29.439 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.440 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:29.500 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:29.495+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:355: done (2 left) 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:356: read oid 5 snap -1 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:356: expect (ObjNum 86 snap 14 seq_num 86) 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:357: delete oid 28 current snap is 33 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 4 to smithi02732947-8 2024-09-15T21:01:29.557 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 5 to smithi02732947-8 2024-09-15T21:01:29.558 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 6 to smithi02732947-8 2024-09-15T21:01:29.558 INFO:tasks.rados.rados.0.smithi027.stdout:354: expect (ObjNum 96 snap 17 seq_num 96) 2024-09-15T21:01:29.590 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.590 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.604 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:29.594+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:29.609 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:29.611 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 2373419 2024-09-15T21:01:29.632 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:29.702 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:29.694+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:29.711 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 2588984 2024-09-15T21:01:29.719 INFO:tasks.rados.rados.0.smithi027.stdout:351: finishing write tid 7 to smithi02732947-8 2024-09-15T21:01:29.719 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 updating version 0 to 80 2024-09-15T21:01:29.719 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 updating version 80 to 84 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 version 84 is already newer than 83 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 version 84 is already newer than 82 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 updating version 84 to 85 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:351: oid 8 version 85 is already newer than 81 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 8 v 85 (ObjNum 141 snap 32 seq_num 141) dirty exists 2024-09-15T21:01:29.720 INFO:tasks.rados.rados.0.smithi027.stdout:351: left oid 8 (ObjNum 141 snap 32 seq_num 141) 2024-09-15T21:01:29.734 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.734 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.749 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:29.803 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:29.795+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:29.876 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:29.876 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:29.890 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:29.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:29.935+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:30.004 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:29.999+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:30.016 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.016 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:30.106 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:30.103+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:30.201 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.201 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.206 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:30.198+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:30.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:30.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:30.303+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:30.329 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.329 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:30.409 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:30.402+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:30.474 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.474 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:30.510 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:30.503+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:30.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:01:30.611 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:30.607+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:30.661 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:30.727 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:30.722+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:30.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:30.803 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.803 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.813 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:30.806+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:30.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:30.914 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:30.907+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:30.942 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:30.942 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:30.959 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:30.959 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:30.977 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:30.977 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:30.985 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:30.985 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:30.993 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:30.994 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:31.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:31.014 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:31.015 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:31.015 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.007+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:31.035 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:31.035 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:31.055 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:31.056 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:31.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:31.080 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:31.100 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:31.115 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:31.111+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:31.119 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:31.123 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:31.123 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:31.123 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:31.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:31.142 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:31.142 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:31.162 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:31.162 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:31.216 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:31.211+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/28 ECSubWrite(tid=18, reqid=client.4641.0:166, at_version=34'9, trim_to=0'0, roll_forward_to=33'8))", 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.149471812000002, 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20249605200000001, 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.271 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115545+0000", 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115546+0000", 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5900000000000005e-07 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115662+0000", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011552699999999999 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115676+0000", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4547e-05 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215777+0000", 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100100516 2024-09-15T21:01:31.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215874+0000", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7094000000000002e-05 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.215886+0000", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1759999999999999e-05 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.317999+0000", 2024-09-15T21:01:31.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102113494 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.318041+0000", 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2154999999999999e-05 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:31.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.891863762, 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:01:31.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:01:31.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=39, reqid=client.4641.0:248, at_version=39'28, trim_to=0'0, roll_forward_to=39'26))", 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.791383678000003, 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20147183599999999, 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473633+0000", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473634+0000", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473665+0000", 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0868000000000003e-05 2024-09-15T21:01:31.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473707+0000", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1860999999999998e-05 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473724+0000", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7206000000000001e-05 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673974+0000", 2024-09-15T21:01:31.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20025005600000001 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.673991+0000", 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6368999999999999e-05 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675017+0000", 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001026267 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.675105+0000", 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8195000000000005e-05 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.6s6 48/28 ECSubWrite(tid=41, reqid=client.4641.0:360, at_version=48'16, trim_to=0'0, roll_forward_to=45'13))", 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.712115924000003, 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20315024000000001, 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552901+0000", 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.552903+0000", 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.561e-06 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553100+0000", 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019769300000000001 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.553123+0000", 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3008999999999999e-05 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.653267+0000", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10014326899999999 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753534+0000", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100267523 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.753550+0000", 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5824999999999998e-05 2024-09-15T21:01:31.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.755996+0000", 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024464930000000001 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:40.756051+0000", 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4867000000000002e-05 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.292 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.501467869999999, 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30705592100000001, 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763551+0000", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763784+0000", 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023310900000000001 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763818+0000", 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3855e-05 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763835+0000", 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6393e-05 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964212+0000", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200376947 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964244+0000", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2035000000000001e-05 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070501+0000", 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10625775599999999 2024-09-15T21:01:31.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070605+0000", 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103706 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=5, reqid=client.4641.0:430, at_version=59'75, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.496193540999997, 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.402708554, 2024-09-15T21:01:31.297 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768826+0000", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000002e-06 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769062+0000", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581599999999999 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769156+0000", 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4553000000000001e-05 2024-09-15T21:01:31.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769216+0000", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9932000000000001e-05 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966630+0000", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19741336600000001 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966643+0000", 2024-09-15T21:01:31.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3609e-05 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171495+0000", 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20485210600000001 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171532+0000", 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6711999999999997e-05 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.393223495999997, 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299792744, 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871820+0000", 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5873000000000001e-05 2024-09-15T21:01:31.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871847+0000", 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6792e-05 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871873+0000", 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6075999999999999e-05 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068219+0000", 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19634641799999999 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068236+0000", 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6912999999999999e-05 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171560+0000", 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10332366 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171586+0000", 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6438999999999999e-05 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 40.392968351999997, 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29954117000000002, 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872055+0000", 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8370000000000002e-06 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872067+0000", 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493e-05 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872079+0000", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1456e-05 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171223+0000", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299144415 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171590+0000", 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036652199999999998 2024-09-15T21:01:31.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=57, reqid=client.4641.0:465, at_version=60'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.464213119999997, 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30393555799999999, 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-07 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800860+0000", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5575000000000002e-05 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800883+0000", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3116e-05 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800903+0000", 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9426e-05 2024-09-15T21:01:31.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001512+0000", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20060972299999999 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001540+0000", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7674999999999998e-05 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104682+0000", 2024-09-15T21:01:31.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103142055 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104739+0000", 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7481000000000001e-05 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=58, reqid=client.4641.0:466, at_version=60'38, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.460349995999998, 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30238109299999999, 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:31.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804669+0000", 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7969999999999999e-06 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804721+0000", 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1820999999999997e-05 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804762+0000", 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1115999999999997e-05 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804786+0000", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4352e-05 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103533+0000", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29874738099999998 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103546+0000", 2024-09-15T21:01:31.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2768e-05 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.106961+0000", 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415094 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107048+0000", 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6763999999999998e-05 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=59, reqid=client.4641.0:467, at_version=60'39, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.454196809999999, 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29628899199999997, 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:31.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810822+0000", 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0430000000000002e-06 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810872+0000", 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9962e-05 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810913+0000", 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0642000000000002e-05 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810940+0000", 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6662e-05 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104296+0000", 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29335650600000002 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104312+0000", 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5559000000000001e-05 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107057+0000", 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027455829999999998 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107109+0000", 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2034999999999999e-05 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.365046040999999, 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20751788099999999, 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899973+0000", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900092+0000", 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118466 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900123+0000", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1149000000000003e-05 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900146+0000", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779000000000001e-05 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900331+0000", 2024-09-15T21:01:31.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000184594 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900362+0000", 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0559000000000003e-05 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107409+0000", 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20704719299999999 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107489+0000", 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0061000000000005e-05 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=16, reqid=client.4641.0:462, at_version=60'83, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.364821192000001, 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20735974700000001, 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.329 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900197+0000", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900419+0000", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022236900000000001 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900447+0000", 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7373e-05 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900472+0000", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981999999999999e-05 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002446+0000", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101974907 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002464+0000", 2024-09-15T21:01:31.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045999999999999e-05 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107496+0000", 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105032797 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107556+0000", 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259000000000003e-05 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.333 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=17, reqid=client.4641.0:463, at_version=60'84, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.364498888999996, 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20715383400000001, 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.334 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900520+0000", 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900569+0000", 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9608999999999999e-05 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900593+0000", 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4034999999999999e-05 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900613+0000", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103732+0000", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.203119718 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103749+0000", 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-09-15T21:01:31.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107564+0000", 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038144160000000002 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107672+0000", 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108294 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 60/28 ECSubWrite(tid=183, reqid=client.4641.0:448, at_version=60'44, trim_to=0'0, roll_forward_to=56'42))", 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 39.356419346999999, 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20201134200000001, 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.908598+0000", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910197+0000", 2024-09-15T21:01:31.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015987340000000001 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910229+0000", 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2623000000000003e-05 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.910247+0000", 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7075000000000002e-05 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104183+0000", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19393659899999999 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104208+0000", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499e-05 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110513+0000", 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0063052070000000002 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.110609+0000", 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6191000000000001e-05 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.1s5 73/71 ECSubWrite(tid=111, reqid=osd.0.0:110, at_version=0'0, trim_to=0'0, roll_forward_to=73'34))", 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.89389031, 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.202398409, 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.343 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371127+0000", 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371129+0000", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6000000000000001e-06 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371161+0000", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1343000000000001e-05 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371203+0000", 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1943000000000001e-05 2024-09-15T21:01:31.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371228+0000", 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5338999999999999e-05 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371405+0000", 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177472 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.371436+0000", 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1016000000000003e-05 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.573382+0000", 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20194593599999999 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.573525+0000", 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014276 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 73/72 ECSubWrite(tid=91, reqid=osd.8.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=73'55))", 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 25.889596921999999, 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29881405799999999, 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375421+0000", 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375434+0000", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2388000000000001e-05 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375447+0000", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2979000000000001e-05 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375456+0000", 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8870000000000004e-06 2024-09-15T21:01:31.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572431+0000", 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19697500200000001 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572450+0000", 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8260000000000001e-05 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674174+0000", 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10172400700000001 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674234+0000", 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0445000000000001e-05 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 79/57 ECSubRead(tid=77, to_read={3:26d7dfe1:::smithi02732947-26:head=0,499712,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:31.352 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.026780448, 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20146718899999999, 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238237+0000", 2024-09-15T21:01:31.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238238+0000", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0219999999999999e-06 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238250+0000", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2455e-05 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.238277+0000", 2024-09-15T21:01:31.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7385e-05 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.338396+0000", 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100118153 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.438594+0000", 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100198367 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:11.439704+0000", 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001109807 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s1 87/84 ECSubWrite(tid=135, reqid=client.4641.0:844, at_version=87'40, trim_to=0'0, roll_forward_to=86'38))", 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.290055046999999, 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.204339415, 2024-09-15T21:01:31.356 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974964+0000", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.367e-06 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975181+0000", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021703299999999999 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975220+0000", 2024-09-15T21:01:31.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.82e-05 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975244+0000", 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4528999999999998e-05 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.175627+0000", 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20038291699999999 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.175660+0000", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3334999999999998e-05 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.179198+0000", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035375179999999999 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.179301+0000", 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103516 2024-09-15T21:01:31.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 87/28 ECSubWrite(tid=355, reqid=client.4641.0:821, at_version=87'78, trim_to=0'0, roll_forward_to=86'75))", 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.9200816730000003, 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20320760800000001, 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344937+0000", 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344980+0000", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2660999999999997e-05 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345013+0000", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3047999999999999e-05 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445166+0000", 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100153248 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445361+0000", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019436099999999999 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445378+0000", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7278999999999998e-05 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.548055+0000", 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102676964 2024-09-15T21:01:31.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.548143+0000", 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7904000000000004e-05 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.365 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:31.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:31.367 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:31.362+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:31.468 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:31.462+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:31.478 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.471+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3 to read 0,1,4,5 2024-09-15T21:01:31.479 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.475+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:01:31.479 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.475+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,4,5 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 97/96 ECSubWriteReply(tid=125, last_complete=97'179, committed=1, applied=1))", 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:31.376562+0000", 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.16850124699999999, 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.168540356, 2024-09-15T21:01:31.546 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376562+0000", 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376562+0000", 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376564+0000", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2729999999999999e-06 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376570+0000", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6270000000000002e-06 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376588+0000", 2024-09-15T21:01:31.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7524999999999999e-05 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:31.376598+0000", 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0686999999999999e-05 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2024-09-15T21:01:31.549 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:31.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:31.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:31.563+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:31.611 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:01:31.674 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:31.666+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:31.730 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:31.730 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:31.730 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:31.730 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:31.730 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:31.740 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:31.735+0000 7f61d0b44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:31.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:31.774 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:31.771+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:31.782 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.775+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,4,5 2024-09-15T21:01:31.784 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.779+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:31.785 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.779+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:01:31.785 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.779+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,7 to read 0,1,4,5 2024-09-15T21:01:31.787 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.779+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,4,5 2024-09-15T21:01:31.787 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.783+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:31.791 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.787+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:01:31.791 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.787+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,4,5 2024-09-15T21:01:31.865 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:31.858+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:31.878 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.871+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:31.892 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:31.887+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,7 to read 0,1,4,5 2024-09-15T21:01:31.895 INFO:tasks.rados.rados.0.smithi027.stdout:356: expect (ObjNum 86 snap 14 seq_num 86) 2024-09-15T21:01:31.915 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1163370 2024-09-15T21:01:31.928 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:31.936 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:31.931+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:430 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~770048] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.327450964000001, 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67855575999999995, 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.942 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 430 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610260+0000", 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611394+0000", 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011336479999999999 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611427+0000", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3259000000000001e-05 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611443+0000", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-05 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.765717+0000", 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15427427099999999 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.766028+0000", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031123899999999999 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768367+0000", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023391079999999999 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858789+0000", 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090422016999999993 2024-09-15T21:01:31.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288681+0000", 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42989205899999999 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288813+0000", 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131722 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:431 3.4s0 3:22a20f07:::smithi02732947-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.326237081000002, 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.77799016399999998, 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.948 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 431 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611472+0000", 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.176e-06 2024-09-15T21:01:31.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611480+0000", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2269999999999996e-06 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611491+0000", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1776e-05 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611500+0000", 2024-09-15T21:01:31.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1929999999999993e-06 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770200+0000", 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15870036200000001 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770513+0000", 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031272599999999999 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.771421+0000", 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090811600000000004 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858791+0000", 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087370619999999996 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389309+0000", 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.53051743900000004 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389461+0000", 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015252899999999999 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:432 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.326195210000002, 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78326424900000002, 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:31.954 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 432 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611514+0000", 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.004e-06 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611517+0000", 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4029999999999999e-06 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611528+0000", 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611536+0000", 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772246+0000", 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.160710251 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772520+0000", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027356700000000001 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390423+0000", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61790300099999995 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390689+0000", 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026639399999999998 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.394777+0000", 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040880530000000003 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:576 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~786432] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.331781966000001, 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61536784200000005, 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 576 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605928+0000", 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607006+0000", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001077862 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607032+0000", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5885999999999999e-05 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607045+0000", 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-15T21:01:31.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662488+0000", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055442577999999999 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662791+0000", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030328799999999997 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.665971+0000", 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031791979999999998 2024-09-15T21:01:31.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816856+0000", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15088531299999999 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221142+0000", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40428599700000001 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221294+0000", 2024-09-15T21:01:31.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152297 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:577 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~376832] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.328475204, 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61247240599999997, 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 577 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609236+0000", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6460000000000002e-06 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610212+0000", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097600200000000001 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610242+0000", 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9963000000000001e-05 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610268+0000", 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712231+0000", 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101962937 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712411+0000", 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018002200000000001 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.714107+0000", 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696896 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816859+0000", 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102751021 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221545+0000", 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40468618099999998 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221705+0000", 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016079899999999999 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:578 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 30.327398915, 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61180690699999996, 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.971 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 578 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610311+0000", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610321+0000", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9690000000000006e-06 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610342+0000", 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0375999999999999e-05 2024-09-15T21:01:31.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610357+0000", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5237e-05 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817100+0000", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20674321900000001 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817311+0000", 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021106099999999999 2024-09-15T21:01:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818686+0000", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013748009999999999 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.919851+0000", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101164695 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221947+0000", 2024-09-15T21:01:31.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30209587999999998 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.222116+0000", 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016968899999999999 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:774 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~573440] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.280353770000001, 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.79679720700000001, 2024-09-15T21:01:31.976 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:31.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657358+0000", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0939999999999999e-06 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658209+0000", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085119500000000001 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658243+0000", 2024-09-15T21:01:31.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3707000000000001e-05 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658259+0000", 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6985000000000001e-05 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837273+0000", 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17901335199999999 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837446+0000", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017337400000000001 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.840435+0000", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029892199999999999 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853927+0000", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013491981 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.453952+0000", 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60002478699999995 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.454152+0000", 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199512 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:775 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.276953107000001, 2024-09-15T21:01:31.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89396708599999997, 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.983 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660758+0000", 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661604+0000", 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084594499999999997 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661636+0000", 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2280000000000003e-05 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661651+0000", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4962e-05 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844564+0000", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18291332699999999 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844791+0000", 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000226646 2024-09-15T21:01:31.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.846827+0000", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020363400000000002 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853934+0000", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071071449999999996 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554542+0000", 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70060748500000003 2024-09-15T21:01:31.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554722+0000", 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018044600000000001 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:776 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.273471013999998, 2024-09-15T21:01:31.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89087423499999996, 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664240+0000", 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4930000000000001e-06 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665080+0000", 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084009500000000001 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665112+0000", 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1961000000000001e-05 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665127+0000", 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5228999999999999e-05 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848176+0000", 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18304937199999999 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848475+0000", 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029845799999999998 2024-09-15T21:01:31.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.850672+0000", 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021974080000000001 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853936+0000", 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032637159999999998 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554980+0000", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70104405299999994 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555112+0000", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013145 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:777 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~311296] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:31.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.271692727000001, 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88944256600000005, 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 777 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:31.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666018+0000", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159999999999999e-06 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666502+0000", 2024-09-15T21:01:31.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048393600000000002 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666534+0000", 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2255000000000002e-05 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766637+0000", 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100103422 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:31.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851711+0000", 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085073812999999998 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851901+0000", 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018974899999999999 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853195+0000", 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012942540000000001 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:31.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.854601+0000", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001405445 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555319+0000", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70071828300000005 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555458+0000", 2024-09-15T21:01:32.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013909299999999999 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:778 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.171034101, 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88938152199999998, 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.001 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766676+0000", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9400000000000001e-06 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766686+0000", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766707+0000", 2024-09-15T21:01:32.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866809+0000", 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100102237 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866970+0000", 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160848 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867191+0000", 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220576 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867924+0000", 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073345699999999997 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868858+0000", 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093380199999999996 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.655861+0000", 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78700323500000002 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.656056+0000", 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019429899999999999 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:779 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:32.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.070854903000001, 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99582967, 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866856+0000", 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.463e-06 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866868+0000", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2398999999999999e-05 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866896+0000", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7370999999999999e-05 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866914+0000", 2024-09-15T21:01:32.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8788999999999999e-05 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868425+0000", 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001510174 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868575+0000", 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150109 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657374+0000", 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78879953300000005 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657536+0000", 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016209800000000001 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.862683+0000", 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.205146734 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 82/57 ECSubWriteReply(tid=99, last_complete=82'157, committed=1, applied=1))", 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:16.051292+0000", 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.886416601000001, 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60594070700000002, 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.051292+0000", 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.051292+0000", 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.051292+0000", 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.051295+0000", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.756e-06 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.051306+0000", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1392999999999999e-05 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.151405+0000", 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10009876299999999 2024-09-15T21:01:32.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.655720+0000", 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50431462800000004 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657232+0000", 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001512727 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:783 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~4751360,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 15.299742676999999, 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.636081864, 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 783 2024-09-15T21:01:32.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637967+0000", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2899999999999999e-06 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637973+0000", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-06 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637991+0000", 2024-09-15T21:01:32.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7825e-05 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.638004+0000", 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3213e-05 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039573+0000", 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40156882999999999 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039782+0000", 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020919100000000001 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.274047+0000", 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.234265375 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:838 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~737280] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:32.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.211107321, 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61883177, 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 838 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:32.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726605+0000", 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7730000000000001e-06 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727781+0000", 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011759159999999999 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727819+0000", 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8733999999999999e-05 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727838+0000", 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8811999999999999e-05 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728035+0000", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019682099999999999 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728056+0000", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0971999999999998e-05 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.731965+0000", 2024-09-15T21:01:32.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039089399999999996 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732120+0000", 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015466000000000001 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.734867+0000", 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027471489999999999 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946633+0000", 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.211766446 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345294+0000", 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39866105299999999 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345433+0000", 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013849399999999999 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:839 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~770048] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.205507322000001, 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61379308200000005, 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 839 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732204+0000", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733438+0000", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001234392 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733471+0000", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3414999999999998e-05 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733491+0000", 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9698999999999999e-05 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837640+0000", 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104148692 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837807+0000", 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167195 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.840395+0000", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025880370000000001 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946894+0000", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106498578 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345806+0000", 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39891273999999999 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345994+0000", 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018767799999999999 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:840 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~540672] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.201914775000001, 2024-09-15T21:01:32.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61059644300000004, 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 840 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735797+0000", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.67e-06 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736891+0000", 2024-09-15T21:01:32.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010939680000000001 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736923+0000", 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1653999999999999e-05 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736939+0000", 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6045999999999998e-05 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943125+0000", 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20618578000000001 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943433+0000", 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030832599999999999 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.945306+0000", 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018727699999999999 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.047848+0000", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102542726 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346245+0000", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29839692899999998 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346390+0000", 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014457399999999999 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:841 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.199998748000001, 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60917734099999998, 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.038 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 841 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737713+0000", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0230000000000001e-06 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738201+0000", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048861200000000003 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738252+0000", 2024-09-15T21:01:32.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0773000000000003e-05 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738276+0000", 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3825e-05 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.148832+0000", 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41055591200000002 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.149003+0000", 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017122200000000001 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.150384+0000", 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013808959999999999 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154322+0000", 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039383689999999997 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346706+0000", 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.192384097 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346887+0000", 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180612 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:842 3.4s0 3:261a50f9:::smithi02732947-12:head [setxattr _header (57) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.199391544999999, 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70825646600000003, 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 842 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738318+0000", 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738328+0000", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7599999999999997e-06 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738350+0000", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1471000000000002e-05 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738394+0000", 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4484000000000003e-05 2024-09-15T21:01:32.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151567+0000", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.413173238 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151818+0000", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025079399999999998 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153019+0000", 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012006740000000001 2024-09-15T21:01:32.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154654+0000", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00163489 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446326+0000", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29167209999999999 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446573+0000", 2024-09-15T21:01:32.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000247328 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:843 3.4s0 3:261a50f9:::smithi02732947-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.199283921999999, 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.91369339800000005, 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 843 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738426+0000", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4169999999999999e-06 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738435+0000", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0100000000000001e-06 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738461+0000", 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6605e-05 2024-09-15T21:01:32.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738477+0000", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5364999999999999e-05 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153751+0000", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41527431799999998 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153990+0000", 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000238698 2024-09-15T21:01:32.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447565+0000", 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29357503600000001 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447832+0000", 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026724700000000002 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.652118+0000", 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20428570200000001 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.054 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:32.055 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:32.031+0000 7fd53cde6640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.055 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 537842 2024-09-15T21:01:32.056 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:31.979+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.056 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:32.051+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:32.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:32.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:32.151+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 98/96 ECSubWriteReply(tid=172, last_complete=97'24, committed=1, applied=1))", 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:32.182449+0000", 2024-09-15T21:01:32.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.004699656, 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047332570000000003, 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182449+0000", 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182449+0000", 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182449+0000", 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8199999999999999e-07 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182453+0000", 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.507e-06 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182464+0000", 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1234e-05 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182472+0000", 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.622e-06 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 98/96 ECSubWriteReply(tid=172, last_complete=97'24, committed=1, applied=1))", 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:32.182519+0000", 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0046289210000000003, 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047370420000000003, 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182519+0000", 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182519+0000", 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182521+0000", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182528+0000", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1960000000000002e-06 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182552+0000", 2024-09-15T21:01:32.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182570+0000", 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8023e-05 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 98/96 ECSubWriteReply(tid=172, last_complete=97'24, committed=1, applied=1))", 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:32.182541+0000", 2024-09-15T21:01:32.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0046070119999999997, 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0047780239999999996, 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182541+0000", 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182541+0000", 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182542+0000", 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5199999999999996e-07 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182546+0000", 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182556+0000", 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9080000000000007e-06 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:32.182564+0000", 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1829999999999997e-06 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.198 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:32.199 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:01:32.199 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:32.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:32.258 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:32.251+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:32.326 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:32.326 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:32.326 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:32.327 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:32.327 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:32.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:32.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:32.355+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:32.460 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:32.455+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:32.468 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:32.468 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.882114881000007, 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.469 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:01:32.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.881407523999997, 2024-09-15T21:01:32.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.479 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.880834381, 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.480 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:01:32.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.780592440000007, 2024-09-15T21:01:32.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:01:32.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:01:32.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:01:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.674334139999999, 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:01:32.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:01:32.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:01:32.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:32.500 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.674221704999994, 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:01:32.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:01:32.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.es0 33/28 ECSubWriteReply(tid=18, last_complete=33'4, committed=1, applied=1))", 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.660264334999994, 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38400288500000002, 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804715+0000", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804716+0000", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804722+0000", 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0689999999999998e-06 2024-09-15T21:01:32.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804740+0000", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7978999999999998e-05 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.904872+0000", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100132263 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188682+0000", 2024-09-15T21:01:32.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28381031600000001 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.188718+0000", 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5540999999999997e-05 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=28, last_complete=34'9, committed=1, applied=1))", 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.024098467000002, 2024-09-15T21:01:32.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39968706100000001, 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440881+0000", 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440882+0000", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1699999999999997e-07 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440886+0000", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8459999999999999e-06 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440911+0000", 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4494999999999998e-05 2024-09-15T21:01:32.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.440924+0000", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3142999999999999e-05 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840527+0000", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39960277900000002 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840568+0000", 2024-09-15T21:01:32.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1081000000000002e-05 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 34/28 ECSubWriteReply(tid=29, last_complete=34'10, committed=1, applied=1))", 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 66.023374946000004, 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39904099900000001, 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441604+0000", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441605+0000", 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:01:32.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441611+0000", 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441652+0000", 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0890999999999999e-05 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.441671+0000", 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9059000000000002e-05 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840619+0000", 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39894794099999997 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.840645+0000", 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6064999999999999e-05 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:498 3.8s0 3:15433346:::smithi02732947-2:head [stat,delete] snapc 11=[11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.815153056, 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40308387200000001, 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 498 2024-09-15T21:01:32.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649826+0000", 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649829+0000", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0460000000000001e-06 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649844+0000", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4397e-05 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649883+0000", 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8804999999999999e-05 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.649901+0000", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8145999999999999e-05 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.730873+0000", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.080971873 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.731287+0000", 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000414008 2024-09-15T21:01:32.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.733618+0000", 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023318610000000002 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:55.735452+0000", 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018332019999999999 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052787+0000", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.31733539399999999 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.052910+0000", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012313999999999999 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:500 3.es0 3:7cf15e4b:::smithi02732947-48:head [copy-from ver 8] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:32.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.824063213999999, 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.72001296999999997, 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 500 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:32.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640919+0000", 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3100000000000001e-06 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640934+0000", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.491e-05 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640969+0000", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4686e-05 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640986+0000", 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7048999999999999e-05 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641119+0000", 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013317 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641136+0000", 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6558999999999998e-05 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.845985+0000", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20484875999999999 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846249+0000", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026397900000000001 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.056421+0000", 2024-09-15T21:01:32.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210172042 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059407+0000", 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029868300000000002 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360789+0000", 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30138199900000001 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360929+0000", 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013967600000000001 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:501 3.es0 3:7cf15e4b:::smithi02732947-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.823952472999999, 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41837276400000001, 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.529 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 501 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641027+0000", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641028+0000", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641035+0000", 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9929999999999998e-06 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641057+0000", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1382e-05 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641079+0000", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1733999999999999e-05 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741365+0000", 2024-09-15T21:01:32.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10028688500000001 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.741386+0000", 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0423000000000001e-05 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846689+0000", 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105302942 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:32.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846826+0000", 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000137503 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.058847+0000", 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21202036799999999 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059116+0000", 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026980899999999998 2024-09-15T21:01:32.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059400+0000", 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028316499999999999 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(osd.0.17:3 3.8s0 3:15433346:::smithi02732947-2:head [assert-version v76,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:32.535 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 16.824359870999999, 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51149464600000005, 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640619+0000", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640622+0000", 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7099999999999999e-06 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640640+0000", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7566000000000001e-05 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640661+0000", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0971e-05 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.640677+0000", 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6226e-05 2024-09-15T21:01:32.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.740946+0000", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100268967 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.740973+0000", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6968e-05 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.041224+0000", 2024-09-15T21:01:32.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30025154500000001 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.041385+0000", 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016017200000000001 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.152114+0000", 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.110729521 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:832 3.8s0 3:15433346:::smithi02732947-2:head [append 0~720896] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.770120613, 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.57907397599999999, 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 832 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.541 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694862+0000", 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2059999999999999e-06 2024-09-15T21:01:32.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696156+0000", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012936379999999999 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696190+0000", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4125000000000003e-05 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696208+0000", 2024-09-15T21:01:32.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8442000000000001e-05 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696374+0000", 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166133 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696390+0000", 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5885000000000001e-05 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749483+0000", 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053093057999999999 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749694+0000", 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021056300000000001 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.752168+0000", 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024746529999999998 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763652+0000", 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011483446 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273791+0000", 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51013918300000005 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273933+0000", 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014164399999999999 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:833 3.8s0 3:15433346:::smithi02732947-2:head [append 0~622592] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.765022252, 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67449799700000002, 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.547 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 833 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:32.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699960+0000", 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.88e-06 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701223+0000", 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012632870000000001 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701257+0000", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4004999999999997e-05 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701280+0000", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3144999999999999e-05 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.754820+0000", 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053539760999999998 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.755101+0000", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028043799999999999 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.758090+0000", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002989317 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763734+0000", 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056437930000000002 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374313+0000", 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61057963000000004 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374455+0000", 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141741 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:834 3.8s0 3:15433346:::smithi02732947-2:head [append 0~688128] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.759080904999999, 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66884350100000001, 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 834 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705901+0000", 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707380+0000", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014797160000000001 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707409+0000", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8951000000000001e-05 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707431+0000", 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1412000000000001e-05 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759507+0000", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052076560000000001 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759651+0000", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014353099999999999 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.762259+0000", 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002607786 2024-09-15T21:01:32.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.864069+0000", 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101810885 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374639+0000", 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51056974700000002 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374742+0000", 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102773 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:835 3.8s0 3:15433346:::smithi02732947-2:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.756882613, 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.667032505, 2024-09-15T21:01:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 835 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708099+0000", 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.553e-06 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708461+0000", 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036148899999999999 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708494+0000", 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3674999999999999e-05 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708517+0000", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2084999999999999e-05 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967220+0000", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25870379199999999 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967394+0000", 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017396100000000001 2024-09-15T21:01:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.968720+0000", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325737 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077387+0000", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10866701600000001 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374989+0000", 2024-09-15T21:01:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29760241199999998 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375129+0000", 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013978499999999999 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:836 3.8s0 3:15433346:::smithi02732947-2:head [setxattr _header (56) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.756425044, 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66703066, 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 836 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708556+0000", 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8670000000000001e-06 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708565+0000", 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5269999999999998e-06 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708590+0000", 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5001e-05 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708606+0000", 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6776e-05 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.969953+0000", 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26134606799999999 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.970101+0000", 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014809999999999999 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971049+0000", 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000948678 2024-09-15T21:01:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077391+0000", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10634163100000001 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375445+0000", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298053911 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375585+0000", 2024-09-15T21:01:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014010100000000001 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:837 3.8s0 3:15433346:::smithi02732947-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.756346279000001, 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87430144200000004, 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 837 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708634+0000", 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.364e-06 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708641+0000", 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9229999999999996e-06 2024-09-15T21:01:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708665+0000", 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3515000000000001e-05 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708681+0000", 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6395000000000001e-05 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971593+0000", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26291159200000003 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971759+0000", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165966 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378011+0000", 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.406252055 2024-09-15T21:01:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378267+0000", 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025595099999999998 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.582935+0000", 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20466768099999999 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:844 3.0s0 3:0196c8aa:::smithi02732947-4:head [create,setxattr lnO\\b_Vrmg\\YoalgS_ap]_oaNZQ[WWtVp\\Pqajab (15856) in=40b,setxattr lk^eORZ`gVcm``\\pr^dth`]NqmVUQkUObOacmWfh (699) in=40b,setxattr alfebafpqnNmSoe_jrs[jlogjdclQl]SfW^_opiR (911) in=40b,setxattr ]tVYcbfsnki]TcRbjSXlkPP\\URmibcZ (559) in=31b,setxattr cS_QsbNZRaNftQNafpmOXrSq[QTT^a`bjr^RjcXe (19716) in=40b,setxattr lWQarRah[_Ri`aaaYU\\YttQj[iZtlmkV]_giaXU] (1119) in=40b,setxattr afP^ZXR]oT[nYcXcXtmS^YhRRO`ah[[igtU\\maam (3576) in=40b,setxattr lfUpnr]dS`jeYdSeReTo^[[scdZRhaXsnpNagRV] (417) in=40b,setxattr oRZflPsTY\\iqWn_qits]aghfipZmOmndmY\\_fdsm (453) in=40b,setxattr cPXrXtr[heq_Y[rbnflZtdiTN[ejbhn]QUcpclY[ (10312) in=40b,setxattr t_S^QjNpSe[mtQWcjpeoQbsQWZqNlnhb`jXVNbkc (1647) in=40b,setxattr QhqWq\\srNqhaWrNUTpj\\lra]SNOZnScUZTX`NXWh (1717) in=40b,setxattr [VXbj[Sc_ttR`ab\\tdcTOlesaQOosdTTQnNtqnS\\ (1329) in=40b,setxattr kOPlXRXq\\neZk[`Qch\\fXP`]t`NWhTlj_XPZhlSY (4560) in=40b,setxattr P^eT`anhlNcnlqOfQsV^QOeZhkX^VbfPm`aNNkSN (1401) in=40b,setxattr gTqoYWiXq`b\\`jd\\V\\URmtOrZernYT_aSTkmR_lV (1715) in=40b,setxattr immbP]\\[qVq`fc\\]bYS\\lmgX[Sk_nfrSS_pTheVl (17540) in=40b,setxattr dRRpo\\[dNWhVPhqhmg[pqo_][oUNQgkOpkpaRPhS (1017) in=40b,setxattr [\\\\ofp^gsRlqNh]W^sW`[[qhU_rb\\qdg\\cffde_g (13964) in=40b,setxattr \\d[kjqp`[PR] (559) in=12b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 11.724604554999999, 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.63608215899999998, 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 844 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740405+0000", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0065000000000001e-05 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740634+0000", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229232 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740675+0000", 2024-09-15T21:01:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0556999999999998e-05 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740697+0000", 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2138999999999999e-05 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971827+0000", 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23113030700000001 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.972183+0000", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035558600000000002 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974826+0000", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026435999999999999 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077486+0000", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102659311 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375591+0000", 2024-09-15T21:01:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298105283 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.376457+0000", 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086607899999999996 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:32.583 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:32.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:32.585 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:32.579+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:32.686 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:32.678+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:32.727 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:919 3.17s0 3.dea8a937 (undecoded) ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e98)", 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:31.663186+0000", 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 1.0628765330000001, 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.062917434, 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "waiting for peered", 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:32.728 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 919 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663186+0000", 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663186+0000", 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663188+0000", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6610000000000001e-06 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663193+0000", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.733e-06 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663206+0000", 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2096000000000001e-05 2024-09-15T21:01:32.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.663217+0000", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1517e-05 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.836676+0000", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.173458847 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.836683+0000", 2024-09-15T21:01:32.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7329999999999997e-06 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.627932+0000", 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79124977100000005 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.627938+0000", 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7880000000000003e-06 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:32.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:32.733 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-15T21:01:32.733 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 1 2024-09-15T21:01:32.733 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:32.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:32.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:32.783+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:32.833 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:32.827+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.843 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:32.838+0000 7fc940c53640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.848 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:32.843+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.869 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:32.870 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:32.870 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:32.870 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:32.870 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:32.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:32.888 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:32.883+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:32.926 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:32.919+0000 7f5af840e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:32.988 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:32.983+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:85 3.1s0 3:81f89603:::smithi02732947-29:head [append 0~475136,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:33.008 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 69.738796078999997, 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39649403999999999, 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 85 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264585+0000", 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264587+0000", 2024-09-15T21:01:33.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.609e-06 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264854+0000", 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026753600000000002 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264865+0000", 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0835e-05 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264870+0000", 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264926+0000", 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5625999999999999e-05 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265044+0000", 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000118676 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.012 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.265921+0000", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00087649699999999995 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367651+0000", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101730565 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.660901+0000", 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.293249816 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661079+0000", 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017783100000000001 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:86 3.1s0 3:81f89603:::smithi02732947-29:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 69.738497940000002, 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39677944799999998, 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 86 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264883+0000", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264884+0000", 2024-09-15T21:01:33.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1999999999999999e-07 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264887+0000", 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1870000000000001e-06 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264894+0000", 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7560000000000002e-06 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264899+0000", 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366599+0000", 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101699788 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.366702+0000", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103047 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367198+0000", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00049636500000000002 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367872+0000", 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000673462 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661403+0000", 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29353088999999999 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.661663+0000", 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00025993499999999998 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:33.020 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 69.738473646000003, 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.021 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:01:33.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.6s0 32/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 69.531411184000007, 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39574710600000002, 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.026 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471970+0000", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471972+0000", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7e-06 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.471997+0000", 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5174e-05 2024-09-15T21:01:33.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472020+0000", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3584999999999999e-05 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.472037+0000", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867596+0000", 2024-09-15T21:01:33.028 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39555944199999998 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.867717+0000", 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000120524 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:254 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 61.878175141, 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.403819759, 2024-09-15T21:01:33.029 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 254 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125206+0000", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125208+0000", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125222+0000", 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3816e-05 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.125254+0000", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1680000000000002e-05 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225338+0000", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100084019 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225409+0000", 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1142000000000004e-05 2024-09-15T21:01:33.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225485+0000", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5417e-05 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226186+0000", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00070119500000000005 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227803+0000", 2024-09-15T21:01:33.033 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001617533 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.528896+0000", 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30109229500000001 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.529026+0000", 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000130152 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:255 3.6s0 3:6f7bd2df:::smithi02732947-21:head [rollback 3] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 61.778010082000002, 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40411628799999999, 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 255 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225371+0000", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225373+0000", 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9589999999999998e-06 2024-09-15T21:01:33.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225380+0000", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7850000000000003e-06 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225393+0000", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3531e-05 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225402+0000", 2024-09-15T21:01:33.037 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7409999999999998e-06 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226613+0000", 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012111050000000001 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.226665+0000", 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2386999999999998e-05 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.038 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227003+0000", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033773100000000002 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227805+0000", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00080205999999999995 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629364+0000", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40155826 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.039 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629487+0000", 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012372899999999999 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 61.777965412, 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.041 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.042 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.043 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:01:33.045 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.860493812999998, 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.046 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.048 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.049 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.050 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:01:33.051 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:33.052 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.860402426, 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:33.053 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.054 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.055 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:01:33.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:01:33.057 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.058 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.696705266000002, 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.059 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:01:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:01:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:01:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:362 3.6s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (56) in=7b] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.595909298999999, 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.35221260100000001, 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.065 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 362 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407472+0000", 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407473+0000", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7999999999999993e-07 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407479+0000", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5899999999999996e-06 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407492+0000", 2024-09-15T21:01:33.067 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2435999999999999e-05 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407501+0000", 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5589999999999994e-06 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412116+0000", 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046143970000000001 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.068 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.412250+0000", 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013454100000000001 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413018+0000", 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00076778700000000003 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413954+0000", 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000935691 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.069 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759576+0000", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34562273999999998 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.759684+0000", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010788 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:33.070 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 52.595848502999999, 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.071 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:01:33.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:01:33.075 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:772 3.1s0 3:81f89603:::smithi02732947-29:head [copy-from ver 76] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.368715549000001, 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.64200396500000001, 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.076 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:33.077 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634669+0000", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4699999999999998e-06 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634686+0000", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7204999999999999e-05 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634729+0000", 2024-09-15T21:01:33.078 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3251999999999999e-05 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634756+0000", 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6271e-05 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634873+0000", 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116989 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:33.079 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634882+0000", 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638177+0000", 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003295035 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638307+0000", 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013013599999999999 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.165965+0000", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52765778900000004 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.171723+0000", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057583169999999998 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276516+0000", 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104793367 2024-09-15T21:01:33.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276669+0000", 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000153046 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:773 3.1s0 3:81f89603:::smithi02732947-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 17.368572112999999, 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.53435976299999999, 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 773 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.083 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634811+0000", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.804e-06 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634818+0000", 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4549999999999998e-06 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634843+0000", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4895000000000001e-05 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634866+0000", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3198e-05 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634983+0000", 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000117088 2024-09-15T21:01:33.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634995+0000", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1912e-05 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638554+0000", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00355902 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638636+0000", 2024-09-15T21:01:33.086 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1588999999999994e-05 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169005+0000", 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.530369117 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.087 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169073+0000", 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7940000000000003e-05 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169169+0000", 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5744999999999999e-05 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.088 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:850 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 1e] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.337743151, 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3063522750000001, 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 850 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.089 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665639+0000", 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8500000000000001e-07 2024-09-15T21:01:33.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665642+0000", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999999e-06 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665651+0000", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4620000000000002e-06 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765755+0000", 2024-09-15T21:01:33.091 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100104183 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766158+0000", 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040273000000000002 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766308+0000", 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015033700000000001 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.092 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766461+0000", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015210000000000001 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371273+0000", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6048122309999999 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371725+0000", 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00045230200000000002 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663049+0000", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29132430799999998 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663312+0000", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026254099999999998 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.665305+0000", 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019926309999999999 2024-09-15T21:01:33.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870050+0000", 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20474537700000001 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971470+0000", 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10142055 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.095 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971990+0000", 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00051970900000000003 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:851 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.237583002999999, 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3075131120000001, 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 851 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.097 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765800+0000", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1660000000000001e-06 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765811+0000", 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0787e-05 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765844+0000", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2492e-05 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765866+0000", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2135999999999999e-05 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769059+0000", 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031937459999999999 2024-09-15T21:01:33.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769218+0000", 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015836100000000001 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371923+0000", 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6027050859999998 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:33.100 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.372518+0000", 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059501300000000003 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666553+0000", 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29403548499999999 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666683+0000", 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129827 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.101 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.667532+0000", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084906999999999995 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870061+0000", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20252906200000001 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.072834+0000", 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202773013 2024-09-15T21:01:33.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.073311+0000", 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047686799999999998 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:902 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~425984] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.404893993, 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50413778399999998, 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 902 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598489+0000", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.685e-06 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598759+0000", 2024-09-15T21:01:33.105 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000270079 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598779+0000", 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0349000000000001e-05 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598790+0000", 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0718e-05 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.106 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.681231+0000", 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.082440988000000007 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.681236+0000", 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.587e-06 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.783400+0000", 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10216322899999999 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.107 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.783592+0000", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019240800000000001 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.784623+0000", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010312710000000001 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.886847+0000", 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102223972 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.102542+0000", 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21569451100000001 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.102625+0000", 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2986999999999997e-05 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:903 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~573440] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.3982937389999996, 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59821814399999995, 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 903 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605090+0000", 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3199999999999998e-06 2024-09-15T21:01:33.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605629+0000", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053903700000000002 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605648+0000", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9008000000000002e-05 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605658+0000", 2024-09-15T21:01:33.112 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0152999999999999e-05 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.887227+0000", 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.281569399 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.887364+0000", 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000136565 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:33.113 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.888691+0000", 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326785 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.892180+0000", 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034895339999999999 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.203243+0000", 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31106281699999999 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.114 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.203305+0000", 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2526000000000004e-05 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 94/93 ECSubWriteReply(tid=158, last_complete=94'53, committed=1, applied=1))", 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.0038548560000002, 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40489114599999998, 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.116 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999527+0000", 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.850000000000001e-07 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999535+0000", 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6650000000000003e-06 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999548+0000", 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3084e-05 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.117 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999558+0000", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0012e-05 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404398+0000", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40484052599999998 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404417+0000", 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 94/93 ECSubWriteReply(tid=157, last_complete=94'52, committed=1, applied=1))", 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:28.000870+0000", 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 5.0025115439999999, 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40344027799999999, 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000870+0000", 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000870+0000", 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000870+0000", 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3500000000000002e-07 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.120 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000873+0000", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.932e-06 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000881+0000", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1349999999999992e-06 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.000887+0000", 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1179999999999997e-06 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404272+0000", 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40338489300000002 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404310+0000", 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7765000000000002e-05 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:33.123 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.031+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:33.125 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:33.119+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:33.184 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:33.226 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:33.222+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:33.299 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:33.299 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:33.299 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:33.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:33.363 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:33.359+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:33.367 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:33.367 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:33.384 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:33.384 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:33.402 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:33.402 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:33.420 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:33.420 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:33.427 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:33.422+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:33.441 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:33.451 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:33.451 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:33.462 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:33.466 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:33.485 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:33.507 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:33.508 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:33.528 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:33.529 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:33.529 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.523+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:33.549 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:33.549 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:33.569 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:33.569 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:33.569 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-15T21:01:33.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:01:33.570 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:33.570 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:33.591 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:33.591 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:33.607 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:33.607 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:33.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:33.630 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:33.622+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.634 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.635 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.636 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.638 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.639 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.640 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.641 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.641 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.641 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.641 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.641 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:01:33.642 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.642 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.642 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.652 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.652 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.623+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.653 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.654 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.655 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.656 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.657 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.658 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.659 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.660 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.661 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.663 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.664 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.665 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.666 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.667 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.668 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.627+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.669 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.670 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.671 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.672 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.673 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.674 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.675 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.676 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.677 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.678 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.679 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.680 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.681 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.682 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.683 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.684 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.685 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.686 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.686 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.686 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.686 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.686 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.631+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.702 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.702 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.702 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.702 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.703 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.704 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.705 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.635+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.706 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.707 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.708 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.709 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.710 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.711 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.712 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.713 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.714 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.715 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.639+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.716 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.717 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.647+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.719 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.720 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.721 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.722 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.725 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.726 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.727 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.728 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.729 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.730 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.731 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.651+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.732 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.733 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.734 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.735 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.736 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.655+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.737 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.738 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.739 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.740 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.741 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.742 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.659+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.743 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.744 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.745 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.746 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.747 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.748 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.749 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.750 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.751 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.752 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.753 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.663+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.754 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.755 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.756 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.757 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.655+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.760 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.760 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.760 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.760 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.760 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.761 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.762 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.762 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.762 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.655+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.763 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.763 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.764 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.767 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.667+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.768 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.769 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.769 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.769 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.769 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.771 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.771 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.771 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.772 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.772 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.773 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.773 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.773 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.775 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.775 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.775 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.775 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.776 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.780 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.781 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.781 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.783 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.783 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.784 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.785 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.785 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.785 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.787 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.789 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.789 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.789 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.789 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.789 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.790 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.791 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.793 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.793 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.793 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.793 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.793 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.794 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.798 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.799 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.800 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.801 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.671+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.802 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.803 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.804 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.805 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.806 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.808 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.809 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.809 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.810 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.811 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.813 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.814 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.815 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.659+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.816 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.817 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.817 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.817 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.817 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.818 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.819 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.819 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.819 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.663+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.819 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.819 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.820 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.820 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.820 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.820 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.821 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.822 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.822 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.822 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.822 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.823 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.824 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.824 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.824 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.824 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.824 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.825 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.825 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.825 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.825 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.826 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.827 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.827 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.827 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.827 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.828 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.829 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.830 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.831 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.832 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.833 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.835 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.836 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.837 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.837 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.837 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.837 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.837 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.838 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.838 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.839 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.839 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.839 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.839 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.839 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.840 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.841 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.842 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.842 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.842 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.842 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.842 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.843 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.844 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.845 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.845 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.845 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.845 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.675+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.846 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.847 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.848 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.848 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.848 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.848 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.848 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.849 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.850 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.850 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.850 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.850 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.851 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.851 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.851 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.852 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.853 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.853 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.853 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.853 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.854 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.855 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.856 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.856 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.856 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.857 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.857 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.857 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.857 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.857 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.858 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.859 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.860 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.860 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.860 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.860 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.861 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.862 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.863 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.864 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.864 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.864 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.864 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.864 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.865 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.866 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.867 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.867 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.867 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.867 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.867 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.868 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.869 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.870 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.871 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.872 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.874 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.731+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.875 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.876 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.877 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.878 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.879 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.880 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.881 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.882 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.883 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.884 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.889 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.890 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.891 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.892 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.893 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.894 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.895 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.896 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.897 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.898 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.900 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.901 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.902 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.903 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.904 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.905 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.906 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.907 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.908 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.909 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.910 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.911 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.912 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.913 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.914 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.915 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.916 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.917 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.918 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.919 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.920 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.735+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.921 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.922 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.923 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.924 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.925 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.925 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.929 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.931 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.932 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.933 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.934 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.935 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.936 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.739+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.937 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.938 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.938 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.938 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.938 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.938 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:33.939 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.939 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.940 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.940 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.940 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.941 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.942 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.943 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.944 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.945 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.945 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.945 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.946 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.946 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.946 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.946 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.947 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.947 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.948 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.948 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.948 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.948 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.948 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.949 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.949 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.949 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.950 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.950 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.950 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.950 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.951 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.951 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.951 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.951 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.951 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.952 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.952 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.952 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.953 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.954 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.955 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.955 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.955 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.955 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.956 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.957 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.957 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.957 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.957 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.957 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.958 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.959 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.960 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.960 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.960 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.960 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.961 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.961 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.961 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.961 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.962 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.963 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.964 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.965 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.966 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.967 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.968 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.969 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.970 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.971 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.972 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.973 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe638a36640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.974 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.975 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.831+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.976 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.863+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:33.979 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.979 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.980 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.981 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.667+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.981 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.981 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.981 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.982 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.982 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.982 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.983 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-15T21:01:33.627660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106365491,"num_objects":61,"num_object_clones":28,"num_object_copies":464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":2,"num_read":328,"num_read_kb":115850,"num_write":622,"num_write_kb":119472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":66986022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1194,"ondisk_log_size":1194,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":774004,"kb_used_data":351152,"kb_used_omap":18,"kb_used_meta":422573,"kb_avail":1131688076,"statfs":{"total":1159641169920,"available":1158848589824,"internally_reserved":0,"allocated":359579648,"data_stored":511886882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":432715067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-62275584,"num_objects":-38,"num_object_clones":-20,"num_object_copies":-304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":-38,"num_whiteouts":-4,"num_read":-121,"num_read_kb":-31469,"num_write":-294,"num_write_kb":-67976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-41,"num_bytes_recovered":-27353088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.080816"},"pg_stats":[{"pgid":"3.19","version":"0'0","reported_seq":2,"reported_epoch":98,"state":"peering","last_fresh":"2024-09-15T21:01:31.865601+0000","last_change":"2024-09-15T21:01:31.662391+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:31.865601+0000","last_undegraded":"2024-09-15T21:01:31.865601+0000","last_fullsized":"2024-09-15T21:01:31.865601+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4,10,6,7,11,2],"acting":[5,3,4,10,6,7,11,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"unknown","last_fresh":"2024-09-15T21:01:31.667548+0000","last_change":"2024-09-15T21:01:32.070760+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:31.667548+0000","last_undegraded":"2024-09-15T21:01:31.667548+0000","last_fullsized":"2024-09-15T21:01:31.667548+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,0,2,8,7,4,5],"acting":[3,9,0,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T21:01:29.542332+0000","last_change":"2024-09-15T21:01:29.542332+0000","last_active":"2024-09-15T21:01:29.542332+0000","last_peered":"2024-09-15T21:01:29.542332+0000","last_clean":"2024-09-15T21:01:29.542332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:29.542332+0000","last_undegraded":"2024-09-15T21:01:29.542332+0000","last_fullsized":"2024-09-15T21:01:29.542332+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_clean_scrub_stamp":"2024-09-15T21:01:29.542332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"unknown","last_fresh":"2024-09-15T21:01:31.631224+0000","last_change":"2024-09-15T21:01:32.145799+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:31.631224+0000","last_undegraded":"2024-09-15T21:01:31.631224+0000","last_fullsized":"2024-09-15T21:01:31.631224+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,3,8,2,10,6],"acting":[5,7,4,3,8,2,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T21:01:29.542332+0000","last_change":"2024-09-15T21:01:29.542332+0000","last_active":"2024-09-15T21:01:29.542332+0000","last_peered":"2024-09-15T21:01:29.542332+0000","last_clean":"2024-09-15T21:01:29.542332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:29.542332+0000","last_undegraded":"2024-09-15T21:01:29.542332+0000","last_fullsized":"2024-09-15T21:01:29.542332+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_clean_scrub_stamp":"2024-09-15T21:01:29.542332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T21:01:29.542332+0000","last_change":"2024-09-15T21:01:29.542332+0000","last_active":"2024-09-15T21:01:29.542332+0000","last_peered":"2024-09-15T21:01:29.542332+0000","last_clean":"2024-09-15T21:01:29.542332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:29.542332+0000","last_undegraded":"2024-09-15T21:01:29.542332+0000","last_fullsized":"2024-09-15T21:01:29.542332+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_clean_scrub_stamp":"2024-09-15T21:01:29.542332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.621954+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:01:31.621954+0000","last_peered":"2024-09-15T21:01:31.621954+0000","last_clean":"2024-09-15T21:01:31.621954+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:01:31.621954+0000","last_undegraded":"2024-09-15T21:01:31.621954+0000","last_fullsized":"2024-09-15T21:01:31.621954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:08.004258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'41","reported_seq":404,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.260039+0000","last_change":"2024-09-15T21:01:31.365604+0000","last_active":"2024-09-15T21:01:31.260039+0000","last_peered":"2024-09-15T21:01:31.260039+0000","last_clean":"2024-09-15T21:01:31.260039+0000","last_became_active":"2024-09-15T21:01:30.958712+0000","last_became_peered":"2024-09-15T21:01:30.958712+0000","last_unstale":"2024-09-15T21:01:31.260039+0000","last_undegraded":"2024-09-15T21:01:31.260039+0000","last_fullsized":"2024-09-15T21:01:31.260039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-15T21:01:41.303811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30650189,"stat_sum":{"num_bytes":1662976,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1914,"num_write":28,"num_write_kb":1784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1236992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":107,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:30.751192+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:01:30.751192+0000","last_peered":"2024-09-15T21:01:30.751192+0000","last_clean":"2024-09-15T21:01:30.751192+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:01:30.751192+0000","last_undegraded":"2024-09-15T21:01:30.751192+0000","last_fullsized":"2024-09-15T21:01:30.751192+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"95'11","reported_seq":383,"reported_epoch":98,"state":"active+clean+scrubbing","last_fresh":"2024-09-15T21:01:32.037603+0000","last_change":"2024-09-15T21:01:32.037275+0000","last_active":"2024-09-15T21:01:32.037603+0000","last_peered":"2024-09-15T21:01:32.037603+0000","last_clean":"2024-09-15T21:01:32.037603+0000","last_became_active":"2024-09-15T21:01:31.161568+0000","last_became_peered":"2024-09-15T21:01:31.161568+0000","last_unstale":"2024-09-15T21:01:32.037603+0000","last_undegraded":"2024-09-15T21:01:32.037603+0000","last_fullsized":"2024-09-15T21:01:32.037603+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.203164189,"stat_sum":{"num_bytes":745472,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1733,"num_write":3,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1851392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:30.670312+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:01:30.670312+0000","last_peered":"2024-09-15T21:01:30.670312+0000","last_clean":"2024-09-15T21:01:30.670312+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:01:30.670312+0000","last_undegraded":"2024-09-15T21:01:30.670312+0000","last_fullsized":"2024-09-15T21:01:30.670312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"97'179","reported_seq":523,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-15T21:01:32.734987+0000","last_change":"2024-09-15T21:01:32.734987+0000","last_active":"2024-09-15T21:01:32.734987+0000","last_peered":"2024-09-15T21:01:32.734987+0000","last_clean":"2024-09-15T21:01:32.734987+0000","last_became_active":"2024-09-15T21:01:31.064590+0000","last_became_peered":"2024-09-15T21:01:31.064590+0000","last_unstale":"2024-09-15T21:01:32.734987+0000","last_undegraded":"2024-09-15T21:01:32.734987+0000","last_fullsized":"2024-09-15T21:01:32.734987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-15T21:02:01.679872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30785238100000001,"stat_sum":{"num_bytes":16801792,"num_objects":7,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":11999,"num_write":44,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[10,11,9,5,7,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":107,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:30.551572+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:01:30.551572+0000","last_peered":"2024-09-15T21:01:30.551572+0000","last_clean":"2024-09-15T21:01:30.551572+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:01:30.551572+0000","last_undegraded":"2024-09-15T21:01:30.551572+0000","last_fullsized":"2024-09-15T21:01:30.551572+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:10.241744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"97'219","reported_seq":622,"reported_epoch":98,"state":"active+clean+scrubbing","last_fresh":"2024-09-15T21:01:31.989531+0000","last_change":"2024-09-15T21:01:31.888897+0000","last_active":"2024-09-15T21:01:31.989531+0000","last_peered":"2024-09-15T21:01:31.989531+0000","last_clean":"2024-09-15T21:01:31.989531+0000","last_became_active":"2024-09-15T21:01:30.973822+0000","last_became_peered":"2024-09-15T21:01:30.973822+0000","last_unstale":"2024-09-15T21:01:31.989531+0000","last_undegraded":"2024-09-15T21:01:31.989531+0000","last_fullsized":"2024-09-15T21:01:31.989531+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10321058700000001,"stat_sum":{"num_bytes":16809984,"num_objects":9,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":33,"num_read_kb":3251,"num_write":86,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6176768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.3","version":"91'46","reported_seq":374,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.370334+0000","last_change":"2024-09-15T21:01:31.370334+0000","last_active":"2024-09-15T21:01:31.370334+0000","last_peered":"2024-09-15T21:01:31.370334+0000","last_clean":"2024-09-15T21:01:31.370334+0000","last_became_active":"2024-09-15T21:01:31.264196+0000","last_became_peered":"2024-09-15T21:01:31.264196+0000","last_unstale":"2024-09-15T21:01:31.370334+0000","last_undegraded":"2024-09-15T21:01:31.370334+0000","last_fullsized":"2024-09-15T21:01:31.370334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025371349999999998,"stat_sum":{"num_bytes":6094848,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":1159,"num_write":19,"num_write_kb":6504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":194,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.630233+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:01:31.630233+0000","last_peered":"2024-09-15T21:01:31.630233+0000","last_clean":"2024-09-15T21:01:31.630233+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:01:31.630233+0000","last_undegraded":"2024-09-15T21:01:31.630233+0000","last_fullsized":"2024-09-15T21:01:31.630233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"97'60","reported_seq":448,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-15T21:01:32.636772+0000","last_change":"2024-09-15T21:01:32.636772+0000","last_active":"2024-09-15T21:01:32.636772+0000","last_peered":"2024-09-15T21:01:32.636772+0000","last_clean":"2024-09-15T21:01:32.636772+0000","last_became_active":"2024-09-15T21:01:31.363252+0000","last_became_peered":"2024-09-15T21:01:31.363252+0000","last_unstale":"2024-09-15T21:01:32.636772+0000","last_undegraded":"2024-09-15T21:01:32.636772+0000","last_fullsized":"2024-09-15T21:01:32.636772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-15T21:02:02.282934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20373577100000001,"stat_sum":{"num_bytes":7675904,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":4789,"num_write":44,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6807552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.637471+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:01:31.637471+0000","last_peered":"2024-09-15T21:01:31.637471+0000","last_clean":"2024-09-15T21:01:31.637471+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:01:31.637471+0000","last_undegraded":"2024-09-15T21:01:31.637471+0000","last_fullsized":"2024-09-15T21:01:31.637471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"94'55","reported_seq":447,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.357026+0000","last_change":"2024-09-15T21:01:31.357026+0000","last_active":"2024-09-15T21:01:31.357026+0000","last_peered":"2024-09-15T21:01:31.357026+0000","last_clean":"2024-09-15T21:01:31.357026+0000","last_became_active":"2024-09-15T21:01:31.056403+0000","last_became_peered":"2024-09-15T21:01:31.056403+0000","last_unstale":"2024-09-15T21:01:31.357026+0000","last_undegraded":"2024-09-15T21:01:31.357026+0000","last_fullsized":"2024-09-15T21:01:31.357026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.09940351,"stat_sum":{"num_bytes":4120576,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":8794,"num_write":23,"num_write_kb":4088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":6905856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.768755+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:01:31.768755+0000","last_peered":"2024-09-15T21:01:31.768755+0000","last_clean":"2024-09-15T21:01:31.768755+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:01:31.768755+0000","last_undegraded":"2024-09-15T21:01:31.768755+0000","last_fullsized":"2024-09-15T21:01:31.768755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'85","reported_seq":523,"reported_epoch":98,"state":"active+clean+snaptrim","last_fresh":"2024-09-15T21:01:31.893318+0000","last_change":"2024-09-15T21:01:31.269734+0000","last_active":"2024-09-15T21:01:31.893318+0000","last_peered":"2024-09-15T21:01:31.893318+0000","last_clean":"2024-09-15T21:01:31.893318+0000","last_became_active":"2024-09-15T21:01:31.166834+0000","last_became_peered":"2024-09-15T21:01:31.166834+0000","last_unstale":"2024-09-15T21:01:31.893318+0000","last_undegraded":"2024-09-15T21:01:31.893318+0000","last_fullsized":"2024-09-15T21:01:31.893318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001069143,"stat_sum":{"num_bytes":8454144,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":11558,"num_write":31,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":13475840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":169,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.725523+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:01:31.725523+0000","last_peered":"2024-09-15T21:01:31.725523+0000","last_clean":"2024-09-15T21:01:31.725523+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:01:31.725523+0000","last_undegraded":"2024-09-15T21:01:31.725523+0000","last_fullsized":"2024-09-15T21:01:31.725523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:32.709368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":289,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.769029+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:01:31.769029+0000","last_peered":"2024-09-15T21:01:31.769029+0000","last_clean":"2024-09-15T21:01:31.769029+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:01:31.769029+0000","last_undegraded":"2024-09-15T21:01:31.769029+0000","last_fullsized":"2024-09-15T21:01:31.769029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"97'24","reported_seq":372,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-15T21:01:32.636388+0000","last_change":"2024-09-15T21:01:32.182306+0000","last_active":"2024-09-15T21:01:32.636388+0000","last_peered":"2024-09-15T21:01:32.636388+0000","last_clean":"2024-09-15T21:01:32.636388+0000","last_became_active":"2024-09-15T21:01:31.267688+0000","last_became_peered":"2024-09-15T21:01:31.267688+0000","last_unstale":"2024-09-15T21:01:32.636388+0000","last_undegraded":"2024-09-15T21:01:32.636388+0000","last_fullsized":"2024-09-15T21:01:32.636388+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30895339199999999,"stat_sum":{"num_bytes":2768896,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":5,"num_write_kb":2808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":90112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.9","version":"94'77","reported_seq":479,"reported_epoch":97,"state":"active+clean+scrubbing","last_fresh":"2024-09-15T21:01:30.868638+0000","last_change":"2024-09-15T21:01:31.568605+0000","last_active":"2024-09-15T21:01:30.868638+0000","last_peered":"2024-09-15T21:01:30.868638+0000","last_clean":"2024-09-15T21:01:30.868638+0000","last_became_active":"2024-09-15T21:01:30.773553+0000","last_became_peered":"2024-09-15T21:01:30.773553+0000","last_unstale":"2024-09-15T21:01:30.868638+0000","last_undegraded":"2024-09-15T21:01:30.868638+0000","last_fullsized":"2024-09-15T21:01:30.868638+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041963230000000001,"stat_sum":{"num_bytes":7061504,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":1631,"num_write":39,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":13443072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.a","version":"81'15","reported_seq":351,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.623393+0000","last_change":"2024-09-15T21:01:29.579975+0000","last_active":"2024-09-15T21:01:31.623393+0000","last_peered":"2024-09-15T21:01:31.623393+0000","last_clean":"2024-09-15T21:01:31.623393+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:01:31.623393+0000","last_undegraded":"2024-09-15T21:01:31.623393+0000","last_fullsized":"2024-09-15T21:01:31.623393+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017957450000000001,"stat_sum":{"num_bytes":4423680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":12576,"num_write":12,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.b","version":"91'98","reported_seq":462,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.617406+0000","last_change":"2024-09-15T21:01:29.650110+0000","last_active":"2024-09-15T21:01:31.617406+0000","last_peered":"2024-09-15T21:01:31.617406+0000","last_clean":"2024-09-15T21:01:31.617406+0000","last_became_active":"2024-09-15T21:00:51.891415+0000","last_became_peered":"2024-09-15T21:00:51.891415+0000","last_unstale":"2024-09-15T21:01:31.617406+0000","last_undegraded":"2024-09-15T21:01:31.617406+0000","last_fullsized":"2024-09-15T21:01:31.617406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:48.379320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001316593,"stat_sum":{"num_bytes":12288000,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":29767,"num_write":48,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.c","version":"91'87","reported_seq":402,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.627524+0000","last_change":"2024-09-15T21:01:29.682519+0000","last_active":"2024-09-15T21:01:31.627524+0000","last_peered":"2024-09-15T21:01:31.627524+0000","last_clean":"2024-09-15T21:01:31.627524+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:01:31.627524+0000","last_undegraded":"2024-09-15T21:01:31.627524+0000","last_fullsized":"2024-09-15T21:01:31.627524+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10114849200000001,"stat_sum":{"num_bytes":10272768,"num_objects":6,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":11640,"num_write":53,"num_write_kb":10576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.d","version":"91'27","reported_seq":351,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.640301+0000","last_change":"2024-09-15T21:01:29.552297+0000","last_active":"2024-09-15T21:01:31.640301+0000","last_peered":"2024-09-15T21:01:31.640301+0000","last_clean":"2024-09-15T21:01:31.640301+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:01:31.640301+0000","last_undegraded":"2024-09-15T21:01:31.640301+0000","last_fullsized":"2024-09-15T21:01:31.640301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001540498,"stat_sum":{"num_bytes":1687552,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":4922,"num_write":30,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.e","version":"81'20","reported_seq":384,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-15T21:01:32.639016+0000","last_change":"2024-09-15T21:01:29.691884+0000","last_active":"2024-09-15T21:01:32.639016+0000","last_peered":"2024-09-15T21:01:32.639016+0000","last_clean":"2024-09-15T21:01:32.639016+0000","last_became_active":"2024-09-15T21:01:25.767274+0000","last_became_peered":"2024-09-15T21:01:25.767274+0000","last_unstale":"2024-09-15T21:01:32.639016+0000","last_undegraded":"2024-09-15T21:01:32.639016+0000","last_fullsized":"2024-09-15T21:01:32.639016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:35.918982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012578450000000001,"stat_sum":{"num_bytes":786432,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":10,"num_read_kb":1257,"num_write":8,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2375680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":313,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-15T21:01:31.639282+0000","last_change":"2024-09-15T21:01:29.551692+0000","last_active":"2024-09-15T21:01:31.639282+0000","last_peered":"2024-09-15T21:01:31.639282+0000","last_clean":"2024-09-15T21:01:31.639282+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:01:31.639282+0000","last_undegraded":"2024-09-15T21:01:31.639282+0000","last_fullsized":"2024-09-15T21:01:31.639282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"peering","last_fresh":"2024-09-15T21:01:31.623783+0000","last_change":"2024-09-15T21:01:31.623783+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:31.623783+0000","last_undegraded":"2024-09-15T21:01:31.623783+0000","last_fullsized":"2024-09-15T21:01:31.623783+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,10,7],"acting":[6,9,5,3,0,2,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,8,11],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"94'55","reported_seq":435,"reported_epoch":97,"state":"activating","last_fresh":"2024-09-15T21:01:31.361282+0000","last_change":"2024-09-15T21:01:31.255784+0000","last_active":"2024-09-15T21:01:30.650913+0000","last_peered":"2024-09-15T21:01:29.014483+0000","last_clean":"2024-09-15T21:01:29.014483+0000","last_became_active":"2024-09-15T21:01:27.782980+0000","last_became_peered":"2024-09-15T21:01:27.782980+0000","last_unstale":"2024-09-15T21:01:31.361282+0000","last_undegraded":"2024-09-15T21:01:31.361282+0000","last_fullsized":"2024-09-15T21:01:31.361282+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":8795,"num_write":24,"num_write_kb":4088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6905856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":["0(0)","1(7)","4(1)","5(4)","6(6)","9(5)","10(2)","11(3)"],"object_location_counts":[{"shards":"0(0),1(7),4(1),5(4),6(6),9(5),10(2),11(3)","objects":3}],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"19","length":"1"},{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T21:01:29.542332+0000","last_change":"2024-09-15T21:01:29.542332+0000","last_active":"2024-09-15T21:01:29.542332+0000","last_peered":"2024-09-15T21:01:29.542332+0000","last_clean":"2024-09-15T21:01:29.542332+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:29.542332+0000","last_undegraded":"2024-09-15T21:01:29.542332+0000","last_fullsized":"2024-09-15T21:01:29.542332+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:01:29.542332+0000","last_clean_scrub_stamp":"2024-09-15T21:01:29.542332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"unknown","last_fresh":"2024-09-15T21:01:31.634753+0000","last_change":"2024-09-15T21:01:31.971358+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:01:31.634753+0000","last_undegraded":"2024-09-15T21:01:31.634753+0000","last_fullsized":"2024-09-15T21:01:31.634753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[7,5,4,2,6,1,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":105775104,"num_objects":57,"num_object_clones":28,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":2,"num_read":252,"num_read_kb":115786,"num_write":497,"num_write_kb":117320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":66985984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":342429696,"data_stored":499281920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1099,"ondisk_log_size":1099,"up":176,"acting":176,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378647,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52584,"kb_used_data":21400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319256,"statfs":{"total":96636764160,"available":96582918144,"internally_reserved":0,"allocated":21913600,"data_stored":31079611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411351,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91376,"kb_used_data":37972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94280464,"statfs":{"total":96636764160,"available":96543195136,"internally_reserved":0,"allocated":38883328,"data_stored":56794843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411350,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71760,"kb_used_data":36412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300080,"statfs":{"total":96636764160,"available":96563281920,"internally_reserved":0,"allocated":37285888,"data_stored":52202683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411349,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57356,"kb_used_data":26172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314484,"statfs":{"total":96636764160,"available":96578031616,"internally_reserved":0,"allocated":26800128,"data_stored":39934938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411349,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62596,"kb_used_data":31412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309244,"statfs":{"total":96636764160,"available":96572665856,"internally_reserved":0,"allocated":32165888,"data_stored":46476794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444054,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64280,"kb_used_data":33088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307560,"statfs":{"total":96636764160,"available":96570941440,"internally_reserved":0,"allocated":33882112,"data_stored":47213530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444055,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69872,"kb_used_data":38680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301968,"statfs":{"total":96636764160,"available":96565215232,"internally_reserved":0,"allocated":39608320,"data_stored":56675290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444054,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92504,"kb_used_data":39108,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94279336,"statfs":{"total":96636764160,"available":96542040064,"internally_reserved":0,"allocated":40046592,"data_stored":57289934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444053,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52084,"kb_used_data":20892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319756,"statfs":{"total":96636764160,"available":96583430144,"internally_reserved":0,"allocated":21393408,"data_stored":29924314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444055,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56528,"kb_used_data":25336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315312,"statfs":{"total":96636764160,"available":96578879488,"internally_reserved":0,"allocated":25944064,"data_stored":38767578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444053,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47440,"kb_used_data":16248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324400,"statfs":{"total":96636764160,"available":96588185600,"internally_reserved":0,"allocated":16637952,"data_stored":20204525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444054,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55624,"kb_used_data":24432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316216,"statfs":{"total":96636764160,"available":96579805184,"internally_reserved":0,"allocated":25018368,"data_stored":35322842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23875584,"data_stored":34607104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15491072,"data_stored":19488768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20758528,"data_stored":30355456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24801280,"data_stored":38051840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20250624,"data_stored":29208576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38887424,"data_stored":56565760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36212736,"data_stored":53096448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30429184,"data_stored":45170688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":25657344,"data_stored":39219200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":32739328,"data_stored":46497792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36130816,"data_stored":51478528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":37195776,"data_stored":55541760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:01:33.983 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.983 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.983 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 26 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.986 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.987 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:33.970+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:33.987 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.987 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.987 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.988 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.989 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.990 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.990 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.990 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.990 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.991 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.992 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.992 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.992 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.993 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.993 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.993 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.994 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.995 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.995 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.995 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.996 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.996 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.996 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.996 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.996 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.997 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.997 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.997 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.997 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.998 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:33.999 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.000 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.000 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.000 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.000 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.000 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:33.931+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.001 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.001 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.002 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.002 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.002 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.002 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.002 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.003 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.003 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.004 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.004 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.004 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.004 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.005 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.005 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.005 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.005 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.006 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.006 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.007 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.007 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.007 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.008 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.008 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.009 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.009 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.009 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.011 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.012 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.013 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.014 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.015 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.016 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.017 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.017 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.017 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.017 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.018 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.019 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.020 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.679+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.021 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.022 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.023 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.024 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.025 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.025 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.025 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.025 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.026 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.027 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.029 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.030 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.031 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.032 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.032 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.032 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.032 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.033 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.034 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.036 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.039 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.041 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.042 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.045 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.047 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.048 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.675+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.050 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.051 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.054 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.055 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.055 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.055 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.055 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.055 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.062 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.062 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.062 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.062 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.063 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.064 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.065 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.066 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.067 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.068 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.069 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.076 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.080 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.081 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.082 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.083 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.084 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.085 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.086 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.087 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.088 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.089 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.090 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.091 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.683+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.092 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.093 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.094 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.095 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.095 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.095 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.095 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.095 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.096 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.097 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.098 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.098 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.098 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.098 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.099 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.100 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.100 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.100 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.100 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.101 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.103 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.104 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.105 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.106 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.107 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.108 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.108 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.108 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.108 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.109 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.109 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.109 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:34.095+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:34.109 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.110 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.111 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.111 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.111 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.111 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.111 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.112 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.113 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.114 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.115 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.116 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.117 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.118 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.679+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.119 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.120 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.124 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.125 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.129 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.144 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.147 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.147 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.147 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.147 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.148 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.151 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.152 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.153 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.687+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.156 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.157 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:34.158 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.158 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.159 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.160 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.164 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.164 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.165 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.166 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.167 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.174 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.177 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.179 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.182 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.184 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.186 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.187 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.687+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.188 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.189 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.190 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.191 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.192 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.193 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.194 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.195 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.196 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.197 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.198 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.199 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.200 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.201 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.202 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.203 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.204 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.205 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.206 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.206 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.691+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.207 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.208 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.209 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.210 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.211 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.212 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.212 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.212 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.212 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.213 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.214 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.215 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.215 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.215 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.215 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.215 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.216 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.216 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.216 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.216 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.217 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.217 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.217 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.217 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.218 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.218 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.219 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.220 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.221 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.691+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.221 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.221 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.221 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.221 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.222 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.222 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.222 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.222 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.223 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.223 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.223 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.223 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.224 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.224 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.224 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.224 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.224 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.225 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.226 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.227 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.228 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.229 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.230 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.231 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.232 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.233 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.234 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.235 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.236 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.237 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.238 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.239 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.240 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.241 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.242 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.243 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.243 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.243 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.695+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.251 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:34.243+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:34.266 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.267 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.268 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.269 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.270 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.271 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.272 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.273 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.274 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.275 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.276 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.695+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.277 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.278 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.279 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.280 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.281 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.282 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.283 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.284 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.285 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.286 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.287 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.699+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.288 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.289 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.290 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.291 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.703+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.707+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.320 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:34.306+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:34.329 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.329 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.330 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.331 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.332 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.333 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.334 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.335 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.336 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.337 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.747+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.363 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.751+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.364 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.372 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:34.374 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:34.374 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:34.390 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.396 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.397 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.398 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.399 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.400 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.401 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.755+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.402 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.403 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.404 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.405 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.406 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.407 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.408 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.409 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.410 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.411 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.412 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.413 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.414 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.759+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.415 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.416 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.417 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.417 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.417 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.417 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.417 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.418 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.419 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.420 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.420 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.420 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.763+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.421 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.783+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af840e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af840e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af840e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.783+0000 7f5af4406640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.425 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.426 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.427 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.428 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.429 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.430 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.431 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.432 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.433 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.433 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.433 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.433 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.434 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.434 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.434 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.434 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.435 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.436 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.436 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.436 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.436 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.437 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.438 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.438 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.438 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.438 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.439 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.440 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.441 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.442 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.443 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.444 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.445 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.446 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.447 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.448 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.449 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.795+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.450 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.451 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.452 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.453 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.795+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.454 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.455 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.456 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.457 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.458 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:34.447+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.459 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.460 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.461 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.462 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.463 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.464 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.465 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.466 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.467 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.468 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.469 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.470 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.471 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.472 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.473 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.474 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.475 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.476 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.477 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.478 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.479 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.480 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.481 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.482 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.483 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.484 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.485 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.486 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.487 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.488 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.489 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.490 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.491 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.492 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.493 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.494 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.495 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.496 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.497 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.498 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.499 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.500 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.501 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.502 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.503 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.504 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.505 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.506 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.507 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.799+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.508 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.509 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.510 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.511 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.512 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.513 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.513 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:34.513 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:34.513 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.513 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.514 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.515 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.516 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.517 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.518 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.799+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.519 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.520 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.521 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.522 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.523 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.524 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.525 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.526 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.527 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.528 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.529 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.529 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.529 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.529 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.529 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.530 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.531 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.531 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.531 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.531 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.531 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.533 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.534 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.535 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.536 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.537 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:34.526+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.538 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.539 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.540 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.541 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.542 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.543 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.544 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.545 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.549 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.549 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.549 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.549 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.550 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.551 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.552 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.553 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.554 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.555 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.556 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.556 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.556 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.556 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.556 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.557 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.558 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.559 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.560 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.561 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.562 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.563 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.564 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.565 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.566 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.567 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.568 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.569 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.570 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.803+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.571 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.572 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.573 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.574 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.577 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.578 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.579 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.580 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.581 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.581 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.581 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.581 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.582 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.583 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.584 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.585 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.586 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.587 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.588 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.589 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.589 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.589 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.589 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.590 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.591 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.592 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.593 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.594 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.595 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.596 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.597 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.597 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.597 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.597 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.597 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.598 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.599 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.600 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.601 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.602 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.603 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.604 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.803+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.606 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.607 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.608 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.609 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.610 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.611 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.612 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.613 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.614 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.615 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.616 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.617 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.618 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.619 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.620 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.621 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.622 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.623 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.624 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.625 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.626 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.807+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.627 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.628 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.629 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.630 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.630 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.630 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.631 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.632 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 3 pgp_num to 26 2024-09-15T21:01:34.632 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.632 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.632 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:34.626+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:34.632 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.633 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.634 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.635 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.636 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.637 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.638 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.639 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.640 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.807+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.641 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.642 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.643 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.644 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.645 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.646 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.647 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.648 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.649 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.650 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.651 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.652 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.653 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.654 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.811+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.656 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.657 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:33.811+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.863+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.705 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.706 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.707 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.708 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.709 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.710 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.711 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.883+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.712 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.713 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.714 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.715 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.716 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.717 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.718 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.719 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.720 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.721 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.722 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.723 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.724 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.725 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.726 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.727 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.728 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.729 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.887+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.730 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.731 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.732 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.903+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.733 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.734 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.735 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.736 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.737 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.738 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.739 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.740 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.741 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.742 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.743 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.744 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.745 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.746 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.747 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.748 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.749 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.753 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.754 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.755 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.907+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.756 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.757 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.758 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.759 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.761 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.762 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.763 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.764 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.765 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.766 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.767 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.768 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.769 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.770 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.772 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.911+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.773 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.774 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.775 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.776 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.777 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.778 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.779 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.780 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.781 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.782 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.915+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.784 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.785 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.786 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.787 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.788 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.789 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.791 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.792 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.793 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.794 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.795 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.796 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.798 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.799 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.800 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.801 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.802 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.802 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.802 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.802 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.923+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.804 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:34.804 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:34.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:351: done (3 left) 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:354: done (2 left) 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:356: done (1 left) 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:357: done (0 left) 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:358: rollback oid 30 current snap is 33 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 30 to 25 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:359: read oid 19 snap -1 2024-09-15T21:01:34.873 INFO:tasks.rados.rados.0.smithi027.stdout:359: expect deleted 2024-09-15T21:01:34.874 INFO:tasks.rados.rados.0.smithi027.stdout:360: rollback oid 4 current snap is 33 2024-09-15T21:01:34.874 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 4 to 24 2024-09-15T21:01:34.874 INFO:tasks.rados.rados.0.smithi027.stdout:361: copy_from oid 44 from oid 46 current snap is 33 2024-09-15T21:01:34.874 INFO:tasks.rados.rados.0.smithi027.stdout:362: snap_remove snap 28 2024-09-15T21:01:34.888 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.889 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.890 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.891 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.892 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.893 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.894 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.895 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.896 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.897 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.898 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.899 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.900 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.901 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.902 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.963+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.903 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.904 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.905 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.906 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.907 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.909 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.910 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.911 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.912 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.913 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.914 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.915 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.916 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.917 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.918 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.919 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.920 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.921 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.922 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.923 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.924 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.925 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.926 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.927 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.928 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.929 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.930 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.931 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.932 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.933 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.934 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.935 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.936 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.937 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.938 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.939 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.940 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.941 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.971+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.942 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.943 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.945 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.946 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.947 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.948 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.949 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.950 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.951 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.952 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.953 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.957 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.958 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.959 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.960 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.961 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.962 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.963 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.964 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.975+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.965 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.966 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.967 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.968 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.969 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.970 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.971 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.972 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.974 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.975 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.976 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.977 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.978 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.979 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.980 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.981 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.982 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.983 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.984 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.985 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.979+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.986 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.987 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.988 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.989 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.990 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.992 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.993 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.994 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.995 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.996 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.997 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.998 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:34.999 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.000 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.001 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.002 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.003 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.004 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.005 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.006 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.983+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.007 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.008 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.009 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.010 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.011 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.012 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.013 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.014 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.015 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.016 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.017 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.018 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.019 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.020 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.021 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.022 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.023 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.024 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.025 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.026 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.027 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.027 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.027 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.987+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.027 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.987+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.987+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.028 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.028 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.028 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.987+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.029 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.030 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.030 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.030 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.987+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.030 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:35.031 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:35.031 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.987+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.031 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.032 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.033 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.033 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.034 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.034 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.035 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.035 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.035 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.036 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.036 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.036 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.036 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.036 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.037 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.038 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.038 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.038 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.038 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.038 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.039 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.039 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.039 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.039 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.039 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.040 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.040 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.040 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.041 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.041 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.041 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.041 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.042 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.043 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.043 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.043 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.043 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.044 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.044 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.045 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.046 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.046 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.046 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.046 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.047 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.048 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.048 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.048 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.048 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.048 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.049 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.049 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.050 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.051 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.051 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.051 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.051 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.052 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.052 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.052 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.052 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.053 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.054 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.054 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.054 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.054 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.054 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.055 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.055 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.055 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.055 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.056 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.056 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.056 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.057 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.057 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.057 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.057 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.057 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.058 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.058 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.059 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.059 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.059 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.059 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.060 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.060 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.060 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.061 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.062 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.062 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.062 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.062 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.062 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.063 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.064 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.064 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.064 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.064 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.065 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.066 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.067 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.067 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.067 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.068 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.068 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.068 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.068 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.068 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.069 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.070 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.070 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.070 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.071 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.072 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.072 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.072 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.072 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.073 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.073 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.073 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.074 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.075 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.076 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.076 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.076 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.076 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.077 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.078 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.079 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.079 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.079 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.079 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.079 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:33.991+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.121 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.991+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.122 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.123 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.124 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.125 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.126 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.127 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.128 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.129 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.130 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.131 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.132 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.133 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.134 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.135 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.995+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.136 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.137 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.138 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.139 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.140 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.141 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.142 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.143 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.144 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.145 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.146 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.147 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.148 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.149 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.150 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.151 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.152 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:33.999+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.153 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.154 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.155 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.156 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.157 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.161 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.162 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.163 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.164 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.165 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.166 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.167 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.168 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.169 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.170 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.171 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.003+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.173 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.174 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.175 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.176 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.177 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.178 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.179 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.180 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.181 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.182 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.183 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.184 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.185 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.186 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.187 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.007+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.195 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:35.182+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:35.219 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.219 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.219 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.220 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.221 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.222 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.223 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.224 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.225 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.226 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.227 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.228 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.229 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.230 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.063+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.231 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.232 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.233 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.234 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.235 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.236 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.237 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.238 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.239 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.240 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.241 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.242 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.243 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.244 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.245 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.246 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.247 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.248 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.249 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.250 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.067+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.251 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.252 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.253 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.254 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.255 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.256 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.257 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.258 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.259 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.260 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.261 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.262 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.263 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.264 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.265 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.266 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.267 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.268 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.269 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.270 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.271 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.071+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.272 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.273 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.274 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.275 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.276 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.277 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.278 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.075+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.107+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.292 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,5,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.295 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.296 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.297 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.298 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.299 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.300 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.301 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.302 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.303 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.304 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.305 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.306 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.307 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.111+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.308 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.309 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.310 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.311 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.312 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.313 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.313 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.313 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:34.115+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,3,5 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.315 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:35.286+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.341 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.342 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.343 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.344 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.345 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.346 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.347 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.348 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.349 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.350 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.351 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.352 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.175+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.353 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.354 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.355 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.356 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.357 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.358 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.359 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.360 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.361 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.362 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.365 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.366 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.367 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.368 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.179+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.369 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.370 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.371 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.372 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.373 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.374 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.375 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.376 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.377 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.378 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.379 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.380 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.381 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.183+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.382 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.187+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.383 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:34.419+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:35.384 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:34.799+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:35.384 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:35.023+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:35.393 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:35.386+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:35.494 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:35.486+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:35.524 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:35.525 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:35.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:35.595 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:35.586+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:35.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:35.634 INFO:tasks.rados.rados.0.smithi027.stdout:362: done (4 left) 2024-09-15T21:01:35.634 INFO:tasks.rados.rados.0.smithi027.stdout:363: snap_remove snap 22 2024-09-15T21:01:35.671 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:35.671 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:35.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:35.735 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:35.731+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:35.771 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:35.771 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:35.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:35.788 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:35.797 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:35.791+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:35.820 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:35.821 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:35.835 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:35.878 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:35.879 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:35.897 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:35.897 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:35.897 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:35.890+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:35.915 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:35.973 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:35.967+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:35.998 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:35.990+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:36.007 INFO:teuthology.orchestra.run.smithi066.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:36.007 INFO:teuthology.orchestra.run.smithi066.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:36.008 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:36.022 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:36.036 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:36.099 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:36.090+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:36.108 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:36.108 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:36.137 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:36.137 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:36.147 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:36.147 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:36.147 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:36.147 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:36.147 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:36.161 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:36.164 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:36.165 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:36.210 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:36.207+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:36.264 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:36.258+0000 7fbc0b111640 -1 osd.1 pg_epoch: 102 pg[3.2s5( v 98'85 (0'0,98'85] local-lis/les=96/97 n=5 ec=28/28 lis/c=96/96 les/c/f=97/97/0 sis=96) [3,9,6,4,5,1,8,10]p3(0) r=5 lpr=96 luod=0'0 crt=98'85 mlcod 98'85 active mbc={} ps=[19~1,1c~3,20~1]] _scan_rollback_obs: queueing trans to clean up obsolete rollback objs 2024-09-15T21:01:36.292 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.127389893, 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.293 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:01:36.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:01:36.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:36.299 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.127347526999998, 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:01:36.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:01:36.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.1269566, 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.305 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:36.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:01:36.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.310 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.126917323000001, 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.311 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.313 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:01:36.314 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.315 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.316 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.087685567999998, 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:01:36.317 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:36.318 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.319 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.320 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:01:36.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.322 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.323 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.086963216000001, 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.326 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:01:36.327 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:01:36.328 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.329 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.086946725000004, 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:01:36.330 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:36.331 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:01:36.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.333 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:36.334 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:01:36.335 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.336 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 71.593759364999997, 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:01:36.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.338 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:01:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.343 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 71.587317901000006, 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.344 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.346 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:01:36.347 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.348 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:01:36.349 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 71.586891499000004, 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.351 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:01:36.352 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:01:36.353 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.354 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.355 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.356 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 56.147584260999999, 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:01:36.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:36.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:36.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:01:36.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.363 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.365 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:656 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~786432] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.431917321, 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88797498799999997, 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.366 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 656 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.367 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858231+0000", 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.097e-06 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859347+0000", 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001116105 2024-09-15T21:01:36.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859370+0000", 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2289999999999998e-05 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859380+0000", 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0771000000000001e-05 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.369 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.015842+0000", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15646217000000001 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.016121+0000", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027814300000000001 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.017565+0000", 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001443937 2024-09-15T21:01:36.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.124892+0000", 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107327906 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.745992+0000", 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62109974199999995 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.371 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746204+0000", 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021182699999999999 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:657 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.424374094000001, 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88093447300000005, 2024-09-15T21:01:36.372 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 657 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:36.373 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865772+0000", 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.865774+0000", 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.374 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866158+0000", 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038427300000000003 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866179+0000", 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0843999999999999e-05 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.375 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.866190+0000", 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0628000000000001e-05 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.018972+0000", 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.152781957 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.019140+0000", 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000168021 2024-09-15T21:01:36.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.021442+0000", 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0023025670000000002 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.125095+0000", 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.1036528 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.377 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746521+0000", 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62142534100000002 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746707+0000", 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000186112 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.378 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:658 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~442368] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.417334916000001, 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.87439944000000003, 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.379 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 658 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.380 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872811+0000", 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.872813+0000", 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2810000000000001e-06 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.873129+0000", 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031647199999999999 2024-09-15T21:01:36.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.873148+0000", 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8893999999999999e-05 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973289+0000", 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100140643 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.382 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.022799+0000", 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.049510586000000002 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.022941+0000", 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014148399999999999 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.023870+0000", 2024-09-15T21:01:36.383 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092914500000000004 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.125098+0000", 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101227734 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747009+0000", 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62191137600000002 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.384 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747211+0000", 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020182500000000001 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:661 3.5s0 3:a8fd5180:::smithi02732947-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:36.385 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.316302452999999, 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.979178514, 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.386 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 661 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:36.387 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973845+0000", 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.388 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973848+0000", 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3340000000000002e-06 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973861+0000", 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.34e-05 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973870+0000", 2024-09-15T21:01:36.389 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535782+0000", 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56191254599999996 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535923+0000", 2024-09-15T21:01:36.390 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140882 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748675+0000", 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21275177100000001 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748830+0000", 2024-09-15T21:01:36.391 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000155682 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.953022+0000", 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20419203999999999 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.392 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:665 3.5s0 3:a4f26d94:::smithi02732947-27:head [append 0~425984] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 25.316265563000002, 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.87733781899999996, 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 665 2024-09-15T21:01:36.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973881+0000", 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973882+0000", 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.974216+0000", 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033487799999999999 2024-09-15T21:01:36.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.974243+0000", 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6738999999999999e-05 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.074353+0000", 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100109805 2024-09-15T21:01:36.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.636146+0000", 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56179291399999998 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.636411+0000", 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026467999999999999 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.397 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.639540+0000", 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031295020000000002 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.747278+0000", 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107738239 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.398 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.850998+0000", 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103719222 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.851219+0000", 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022109399999999999 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.399 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:764 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~507904 in=507904b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.658935808999999, 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89954056699999996, 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.400 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 764 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:36.401 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631213+0000", 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2170000000000001e-06 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631857+0000", 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00064422299999999997 2024-09-15T21:01:36.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631889+0000", 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2246999999999998e-05 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631905+0000", 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5336e-05 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.403 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320301+0000", 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68839636000000004 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320584+0000", 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028351100000000002 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.404 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.322686+0000", 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021011860000000001 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.326185+0000", 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003499149 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.405 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530672+0000", 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20448741300000001 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530751+0000", 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8924999999999994e-05 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:765 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~475136 in=475136b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.656253939999999, 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0983793850000001, 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 765 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.408 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633894+0000", 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2249999999999999e-06 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634374+0000", 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048002400000000001 2024-09-15T21:01:36.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634405+0000", 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1189999999999998e-05 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634421+0000", 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5815999999999999e-05 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.410 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324182+0000", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68976111799999995 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324311+0000", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000129271 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325308+0000", 2024-09-15T21:01:36.411 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099730799999999992 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.430267+0000", 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10495866700000001 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732131+0000", 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30186411099999999 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.412 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732272+0000", 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140655 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:766 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~98304] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.655617171999999, 2024-09-15T21:01:36.413 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0981193570000001, 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 766 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.414 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.415 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634531+0000", 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4500000000000001e-06 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634708+0000", 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000177104 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634721+0000", 2024-09-15T21:01:36.416 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.359e-05 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634730+0000", 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9020000000000005e-06 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427504+0000", 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.79277363899999997 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427699+0000", 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019504099999999999 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.429045+0000", 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001346564 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.631182+0000", 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202136547 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732474+0000", 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101292148 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732649+0000", 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000174372 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:767 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:36.420 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.655396712000002, 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.709944165, 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 767 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4499999999999996e-07 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634759+0000", 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8650000000000007e-06 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634776+0000", 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6549e-05 2024-09-15T21:01:36.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634783+0000", 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5340000000000002e-06 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935746+0000", 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3009621600000001 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935934+0000", 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000188035 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.936723+0000", 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00078927499999999998 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.143378+0000", 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20665556199999999 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344471+0000", 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20109264199999999 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344694+0000", 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022269799999999999 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.426 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.427 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.427 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.427 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.427 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:36.427 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:36.354+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:36.427 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:36.429 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:36.387+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:36.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:36.471+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:36.561 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:36.555+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:36.566 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:36.566 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.14s5 102/99 [PushOp(3:2b54c2df:::smithi02732947-23:head, version: 71'132, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:2b54c2df:::smithi02732947-23:head@71'132, size: 3555328, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.529632+0000", 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.036012871000000002, 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.036098989999999997, 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529632+0000", 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529632+0000", 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529634+0000", 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.826e-06 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529780+0000", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000146207 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529840+0000", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9726999999999999e-05 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.529867+0000", 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6744999999999999e-05 2024-09-15T21:01:36.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.570 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.570 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.570 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.570 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:36.570 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:01:36.571 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:36.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:363: done (4 left) 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:364: setattr oid 45 current snap is 33 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:360: finishing rollback tid 0 to smithi02732947-4 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:360: finishing rollback tid 1 to smithi02732947-4 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:360: finishing rollback tid 2 to smithi02732947-4 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 4 v 66 (ObjNum 78 snap 10 seq_num 78) dirty exists 2024-09-15T21:01:36.639 INFO:tasks.rados.rados.0.smithi027.stdout:361: finishing copy_from to smithi02732947-44 2024-09-15T21:01:36.640 INFO:tasks.rados.rados.0.smithi027.stdout:361: got expected ENOENT (src dne) 2024-09-15T21:01:36.640 INFO:tasks.rados.rados.0.smithi027.stdout:361: finishing copy_from racing read to smithi02732947-44 2024-09-15T21:01:36.643 INFO:tasks.rados.rados.0.smithi027.stdout:359: done (4 left) 2024-09-15T21:01:36.643 INFO:tasks.rados.rados.0.smithi027.stdout:360: done (3 left) 2024-09-15T21:01:36.644 INFO:tasks.rados.rados.0.smithi027.stdout:361: done (2 left) 2024-09-15T21:01:36.644 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 39 current snap is 33 2024-09-15T21:01:36.644 INFO:tasks.rados.rados.0.smithi027.stdout:365: seq_num 143 ranges {638976=458752,1097728=475136,1572864=655360,2228224=671744,2899968=147456} 2024-09-15T21:01:36.661 INFO:tasks.rados.rados.0.smithi027.stdout:365: writing smithi02732947-39 from 638976 to 1097728 tid 1 2024-09-15T21:01:36.662 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:36.654+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:36.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:01:36.669 INFO:tasks.rados.rados.0.smithi027.stdout:365: writing smithi02732947-39 from 1097728 to 1572864 tid 2 2024-09-15T21:01:36.682 INFO:tasks.rados.rados.0.smithi027.stdout:365: writing smithi02732947-39 from 1572864 to 2228224 tid 3 2024-09-15T21:01:36.695 INFO:tasks.rados.rados.0.smithi027.stdout:365: writing smithi02732947-39 from 2228224 to 2899968 tid 4 2024-09-15T21:01:36.698 INFO:tasks.rados.rados.0.smithi027.stdout:365: writing smithi02732947-39 from 2899968 to 3047424 tid 5 2024-09-15T21:01:36.698 INFO:tasks.rados.rados.0.smithi027.stdout:366: copy_from oid 50 from oid 33 current snap is 33 2024-09-15T21:01:36.698 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 16 current snap is 33 2024-09-15T21:01:36.698 INFO:tasks.rados.rados.0.smithi027.stdout:367: seq_num 144 ranges {4292608=16384} 2024-09-15T21:01:36.738 INFO:tasks.rados.rados.0.smithi027.stdout:367: writing smithi02732947-16 from 4292608 to 4308992 tid 1 2024-09-15T21:01:36.738 INFO:tasks.rados.rados.0.smithi027.stdout:368: rmattr oid 27 current snap is 33 2024-09-15T21:01:36.738 INFO:tasks.rados.rados.0.smithi027.stdout:368: done (5 left) 2024-09-15T21:01:36.738 INFO:tasks.rados.rados.0.smithi027.stdout:369: delete oid 4 current snap is 33 2024-09-15T21:01:36.738 INFO:tasks.rados.rados.0.smithi027.stdout:358: finishing rollback tid 0 to smithi02732947-30 2024-09-15T21:01:36.745 INFO:tasks.rados.rados.0.smithi027.stdout:358: finishing rollback tid 1 to smithi02732947-30 2024-09-15T21:01:36.749 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:36.749 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:36.749 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:36.749 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:36.749 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:36.762 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:36.758+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:36.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:36.863 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:36.862+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.614043503000005, 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.900 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:01:36.901 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.902 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.903 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.612313010999998, 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.904 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.905 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.906 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:01:36.907 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.611065432999993, 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.908 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:01:36.910 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:01:36.911 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.597181328999994, 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.912 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.913 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.914 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.915 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.513299329000006, 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.916 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:36.917 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.918 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.919 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:36.920 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.410852380999998, 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:01:36.922 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:01:36.923 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.409933483000003, 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.925 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.926 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.927 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.407410213999995, 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.928 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.929 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:01:36.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.932 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=37, reqid=client.4641.0:137, at_version=33'14, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 72.405113176, 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295638283, 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489380+0000", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489381+0000", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.02e-06 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489405+0000", 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4046999999999999e-05 2024-09-15T21:01:36.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489426+0000", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0506e-05 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.489438+0000", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1596999999999999e-05 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783625+0000", 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29418769900000002 2024-09-15T21:01:36.935 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783642+0000", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6269999999999998e-05 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784960+0000", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013181379999999999 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.785019+0000", 2024-09-15T21:01:36.936 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.9005999999999997e-05 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 67.315516399000003, 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.937 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.938 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.939 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:01:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.941 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 67.312603718999995, 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.942 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.943 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:01:36.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:01:36.945 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 56.431054435999997, 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.946 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.947 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.948 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:36.949 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.025933035000001, 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:36.950 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.951 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.952 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.953 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:502 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.254488070000001, 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30380249300000001, 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 502 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.954 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640008+0000", 2024-09-15T21:01:36.955 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8420000000000001e-06 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640020+0000", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2067e-05 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640045+0000", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4406999999999999e-05 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640057+0000", 2024-09-15T21:01:36.956 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1917e-05 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640154+0000", 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6767000000000004e-05 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640165+0000", 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.137e-05 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.957 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841240+0000", 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20107475499999999 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841489+0000", 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024974399999999999 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.943808+0000", 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102318624 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:503 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.25441447, 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308825238, 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 503 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640080+0000", 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4600000000000005e-07 2024-09-15T21:01:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640085+0000", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9250000000000003e-06 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640092+0000", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3019999999999997e-06 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640096+0000", 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1919999999999998e-06 2024-09-15T21:01:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740388+0000", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100292057 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740400+0000", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1420000000000001e-05 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845387+0000", 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104987575 2024-09-15T21:01:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845666+0000", 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000279158 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.948904+0000", 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103238063 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:504 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 40.254365958000001, 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409270668, 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 504 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.46e-07 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640133+0000", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4739999999999999e-06 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640145+0000", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2673e-05 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640151+0000", 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2829999999999998e-06 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740469+0000", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100317608 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740477+0000", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2719999999999997e-06 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.846778+0000", 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106300532 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.847035+0000", 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025722499999999998 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:57.049398+0000", 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20236315499999999 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 74/59 ECSubWrite(tid=259, reqid=client.4641.0:614, at_version=74'133, trim_to=0'0, roll_forward_to=73'130))", 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 31.192631064, 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20517745600000001, 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701863+0000", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.701865+0000", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4870000000000001e-06 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702223+0000", 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000357544 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.702264+0000", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1263000000000001e-05 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802363+0000", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10009968800000001 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802532+0000", 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000168856 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.802561+0000", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8552e-05 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.906960+0000", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104399596 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:05.907040+0000", 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9469999999999996e-05 2024-09-15T21:01:36.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:770 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.259724458000001, 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30194747500000002, 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 770 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634770+0000", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6300000000000004e-07 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634775+0000", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8219999999999997e-06 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634791+0000", 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5994999999999998e-05 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734865+0000", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10007408399999999 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735010+0000", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014533800000000001 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735283+0000", 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027298399999999999 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.936717+0000", 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20143348899999999 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:771 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 21.159601167999998, 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21090594100000001, 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.977 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 771 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734892+0000", 2024-09-15T21:01:36.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734894+0000", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.658e-06 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734901+0000", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.2799999999999998e-06 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734910+0000", 2024-09-15T21:01:36.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7290000000000006e-06 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734915+0000", 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6859999999999997e-06 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.736101+0000", 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001186161 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:36.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.736208+0000", 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010704099999999999 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.945798+0000", 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20959038599999999 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 94/93 ECSubWrite(tid=157, reqid=client.4641.0:904, at_version=94'52, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:27.894179+0000", 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 9.0003149350000005, 2024-09-15T21:01:36.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20535806500000001, 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.894179+0000", 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.894179+0000", 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.894180+0000", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.127e-06 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.894338+0000", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015840799999999999 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.894377+0000", 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9323000000000003e-05 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.994490+0000", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011281800000001 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.995481+0000", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099029200000000004 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:27.995503+0000", 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2676000000000001e-05 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:28.099449+0000", 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103945685 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:28.099537+0000", 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7736000000000002e-05 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:36.985 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:36.986 INFO:tasks.rados.rados.0.smithi027.stdout:358: finishing rollback tid 2 to smithi02732947-30 2024-09-15T21:01:36.986 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 30 v 181 (ObjNum 124 snap 23 seq_num 124) dirty exists 2024-09-15T21:01:36.987 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:36.978+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:36.987 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:37.004 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:36.998+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:37.088 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:37.082+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:37.128 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.12869372600000001, 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.12872567600000001, 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996939+0000", 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3e-06 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997370+0000", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043102499999999997 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997402+0000", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2592999999999998e-05 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997419+0000", 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6670000000000001e-05 2024-09-15T21:01:37.131 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.127973954, 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.12807839600000001, 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997658+0000", 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998063+0000", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040452199999999999 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998087+0000", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4320000000000001e-05 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998105+0000", 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7938999999999999e-05 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.134 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.000416+0000", 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.12521526999999999, 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.12538199799999999, 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000416+0000", 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000416+0000", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000416+0000", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000416+0000", 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000468+0000", 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1643e-05 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000488+0000", 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0057000000000001e-05 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.137 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 103/28 ECSubWriteReply(tid=141, last_complete=103'29, committed=1, applied=1))", 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.000908+0000", 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.124722892, 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.124954226, 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000908+0000", 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000908+0000", 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000910+0000", 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000920+0000", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1600000000000004e-06 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000942+0000", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2240000000000001e-05 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000955+0000", 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3689999999999999e-05 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.140 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.124690523, 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.124982473, 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000942+0000", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2780000000000001e-06 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001218+0000", 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027612999999999997 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001230+0000", 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.201e-05 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001237+0000", 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7750000000000004e-06 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.143 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.039737976000000001, 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.040094011999999998, 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085895+0000", 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.145 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086206+0000", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031062599999999999 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086231+0000", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5363000000000001e-05 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086248+0000", 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.719e-05 2024-09-15T21:01:37.146 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:941 3.15s2 3:a8fd5180:::smithi02732947-50:head [copy-from ver 67 in=55b] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.697605+0000", 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.42802599400000002, 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.42849329899999999, 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "waiting for missing object", 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.147 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 941 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.697605+0000", 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.697605+0000", 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.697608+0000", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8700000000000001e-06 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.697622+0000", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4093e-05 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.697655+0000", 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2252999999999998e-05 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797757+0000", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100102018 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.891027+0000", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093270820000000004 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894432+0000", 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034047539999999999 2024-09-15T21:01:37.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s4 103/96 ECSubWrite(tid=394, reqid=osd.5.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=103'223))", 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.039355692999999997, 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.039921980000000003, 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086277+0000", 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.443e-06 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086302+0000", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4885999999999999e-05 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086321+0000", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8870000000000001e-05 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086340+0000", 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8836000000000001e-05 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.153 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 103/28 ECSubWriteReply(tid=141, last_complete=103'29, committed=1, applied=1))", 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.088625+0000", 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.037005820000000002, 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.037635556000000001, 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088625+0000", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088625+0000", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088628+0000", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.08e-06 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088640+0000", 2024-09-15T21:01:37.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2296000000000001e-05 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088667+0000", 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7657e-05 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088681+0000", 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3066e-05 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 103/28 ECSubWriteReply(tid=141, last_complete=103'29, committed=1, applied=1))", 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.088997+0000", 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.036634093999999999, 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.037325496, 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088997+0000", 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088997+0000", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.088999+0000", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6029999999999999e-06 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.089007+0000", 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3399999999999998e-06 2024-09-15T21:01:37.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.089025+0000", 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7856999999999999e-05 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.089043+0000", 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8031999999999999e-05 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.159 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:942 3.15s2 3:a8fd5180:::smithi02732947-50:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.797793+0000", 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.32783800400000002, 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32860216799999997, 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "waiting for missing object", 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 942 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797793+0000", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797793+0000", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797795+0000", 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797804+0000", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7800000000000006e-06 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797825+0000", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.139e-05 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797843+0000", 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7846000000000002e-05 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894712+0000", 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.096869031999999994 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895508+0000", 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00079556400000000002 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:37.163 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.030039586, 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.030890022999999999, 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095593+0000", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.133e-06 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095748+0000", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015470100000000001 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095763+0000", 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299999999999999e-05 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095770+0000", 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5970000000000003e-06 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.166 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:943 3.15s2 3:ad5ec547:::smithi02732947-16:head [append 0~16384 in=16384b] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.797887+0000", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.32774444200000002, 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32866563500000001, 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "waiting for missing object", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 943 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797887+0000", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797887+0000", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797888+0000", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3370000000000001e-06 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797914+0000", 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6033e-05 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797932+0000", 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8162999999999999e-05 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797945+0000", 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2547e-05 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.169 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895760+0000", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.097815247999999994 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900178+0000", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044177319999999997 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:37.170 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.028703988999999999, 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029709301, 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:37.171 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096929+0000", 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8199999999999999e-06 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097251+0000", 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032222400000000002 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.172 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097287+0000", 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6096000000000001e-05 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097302+0000", 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5053e-05 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:944 3.15s2 3:ad5ec547:::smithi02732947-16:head [setxattr _header (57) in=64b] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.797963+0000", 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.32766827300000001, 2024-09-15T21:01:37.173 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32874335100000002, 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "waiting for missing object", 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 944 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797963+0000", 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.174 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797963+0000", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797964+0000", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8400000000000003e-07 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797968+0000", 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2599999999999999e-06 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.175 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797980+0000", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2313000000000001e-05 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.797990+0000", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9830000000000008e-06 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.100931+0000", 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30294030700000002 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.101242+0000", 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000311243 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 103/28 ECSubWriteReply(tid=141, last_complete=103'29, committed=1, applied=1))", 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.097332+0000", 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.028299259, 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.029461283000000001, 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097332+0000", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097332+0000", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.178 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097333+0000", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097338+0000", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.6330000000000001e-06 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097353+0000", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5026e-05 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097362+0000", 2024-09-15T21:01:37.179 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0870000000000007e-06 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:945 3.15s2 3.e2a37ab5 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.798004+0000", 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.32762713300000001, 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.328852912, 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.180 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 945 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798004+0000", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798004+0000", 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.181 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798005+0000", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7300000000000005e-07 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798008+0000", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6740000000000001e-06 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798020+0000", 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2133000000000001e-05 2024-09-15T21:01:37.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.798033+0000", 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2937e-05 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.23109141, 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.232386388, 2024-09-15T21:01:37.183 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.184 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894542+0000", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895468+0000", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092661499999999999 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895495+0000", 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.686e-05 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.185 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.995597+0000", 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100101948 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=1))", 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.22523792500000001, 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.226598313, 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900395+0000", 2024-09-15T21:01:37.187 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.553e-06 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900696+0000", 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030154100000000001 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900721+0000", 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5018999999999999e-05 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.188 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000802+0000", 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100080497 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.14039084800000001, 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.14181080500000001, 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.189 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985242+0000", 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.686e-06 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.190 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985679+0000", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043683900000000001 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985702+0000", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3339000000000001e-05 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085831+0000", 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012874200000001 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.191 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.192 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.192 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:37.192 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 20 2024-09-15T21:01:37.192 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:37.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:37.194 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:37.186+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:37.246 INFO:tasks.rados.rados.0.smithi027.stdout:358: done (5 left) 2024-09-15T21:01:37.246 INFO:tasks.rados.rados.0.smithi027.stdout:369: done (4 left) 2024-09-15T21:01:37.246 INFO:tasks.rados.rados.0.smithi027.stdout:370: snap_remove snap 24 2024-09-15T21:01:37.294 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:37.290+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:37.329 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:37.330 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:37.330 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:37.330 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:37.330 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:37.345 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:37.395 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:37.391+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-15T21:01:37.478 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 98.242263464000004, 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50456940299999997, 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 25 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231729+0000", 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231730+0000", 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231737+0000", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0029999999999997e-06 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231751+0000", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.385e-05 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.231759+0000", 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8499999999999994e-06 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332317+0000", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10055755399999999 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332481+0000", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016460199999999999 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.332691+0000", 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000210039 2024-09-15T21:01:37.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.333716+0000", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001024608 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.735986+0000", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.402270187 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736174+0000", 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018833500000000001 2024-09-15T21:01:37.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T20:59:59.736299+0000", 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000124332 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036987452999995, 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40486488999999998, 2024-09-15T21:01:37.484 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 47 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437005+0000", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437008+0000", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5969999999999999e-06 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437266+0000", 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025848499999999998 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437291+0000", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4576e-05 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437304+0000", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2903e-05 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446656+0000", 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0093519559999999998 2024-09-15T21:01:37.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446766+0000", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010997000000000001 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.446972+0000", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020603 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.741478+0000", 2024-09-15T21:01:37.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29450653500000001 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841727+0000", 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100249009 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841805+0000", 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.7273000000000004e-05 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841870+0000", 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5555999999999994e-05 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.490 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036662292000003, 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.404668058, 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 48 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437330+0000", 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437332+0000", 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2300000000000001e-06 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437338+0000", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7669999999999998e-06 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437349+0000", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0859999999999999e-05 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437357+0000", 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9270000000000005e-06 2024-09-15T21:01:37.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637279+0000", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19992201100000001 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637413+0000", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000133795 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.637619+0000", 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000205689 2024-09-15T21:01:37.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841706+0000", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20408770100000001 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841887+0000", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018052899999999999 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841948+0000", 2024-09-15T21:01:37.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1013e-05 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.841998+0000", 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0535999999999999e-05 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036621435000001, 2024-09-15T21:01:37.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40645446400000002, 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 49 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437371+0000", 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437372+0000", 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:37.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437378+0000", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4999999999999999e-06 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437388+0000", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0077e-05 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437395+0000", 2024-09-15T21:01:37.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3200000000000002e-06 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639349+0000", 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20195405599999999 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639496+0000", 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014738800000000001 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.639693+0000", 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019659199999999999 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842184+0000", 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202491474 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843684+0000", 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0014995189999999999 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843756+0000", 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.1891999999999995e-05 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843826+0000", 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.9809999999999997e-05 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.502 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036572688999996, 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41084704999999999, 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 50 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:37.503 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437420+0000", 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437421+0000", 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.504 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437637+0000", 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021613900000000001 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437675+0000", 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8039e-05 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437694+0000", 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842260+0000", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40456571499999999 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842388+0000", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012835599999999999 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.842614+0000", 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022584800000000001 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846456+0000", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0038414880000000001 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848112+0000", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016563999999999999 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848199+0000", 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7175999999999993e-05 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.848267+0000", 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7729000000000002e-05 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036257778000007, 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.508 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:01:37.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 97.036179712000006, 2024-09-15T21:01:37.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:01:37.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:01:37.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 93.245724007000007, 2024-09-15T21:01:37.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:37.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:01:37.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 93.245553963999996, 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:01:37.526 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:01:37.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:01:37.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:01:37.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:437 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~540672] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.845134643000002, 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54912561999999998, 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 437 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628861+0000", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7499999999999999e-06 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629602+0000", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142900000000005 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629633+0000", 2024-09-15T21:01:37.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0453000000000001e-05 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629649+0000", 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5923e-05 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629768+0000", 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119453 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.630003+0000", 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023483799999999999 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.632434+0000", 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024312940000000001 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.636553+0000", 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041183879999999997 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177838+0000", 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54128565900000003 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177984+0000", 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014543299999999999 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:438 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~229376] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:37.537 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.842565946999997, 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54706032999999998, 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 438 2024-09-15T21:01:37.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631429+0000", 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2630000000000002e-06 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631860+0000", 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000431261 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631886+0000", 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6230000000000001e-05 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631907+0000", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633475+0000", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015675870000000001 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633636+0000", 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016115899999999999 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.635458+0000", 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018221299999999999 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.637212+0000", 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017535420000000001 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178395+0000", 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54118301599999996 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178487+0000", 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2262999999999994e-05 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:439 3.ds0 3:b220672c:::smithi02732947-15:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:37.543 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.842039446999998, 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.85063377100000004, 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 439 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631956+0000", 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-06 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631966+0000", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0047000000000001e-05 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631987+0000", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1253000000000001e-05 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732106+0000", 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011937999999999 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179003+0000", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44689677 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179116+0000", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112794 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179656+0000", 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054039500000000005 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180543+0000", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088661899999999995 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482437+0000", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30189397699999998 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482587+0000", 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015026300000000001 2024-09-15T21:01:37.548 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:440 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 46.741842525000003, 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.75401734600000003, 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.549 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 440 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732153+0000", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.717e-06 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732167+0000", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.382e-05 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732202+0000", 2024-09-15T21:01:37.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5553000000000003e-05 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732225+0000", 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2894000000000001e-05 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180004+0000", 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44777894899999998 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180092+0000", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7705000000000001e-05 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483149+0000", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30305710299999999 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483259+0000", 2024-09-15T21:01:37.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109998 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.486168+0000", 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0029086070000000001 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:652 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.648660450000001, 2024-09-15T21:01:37.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50904735300000004, 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 652 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825336+0000", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2660000000000002e-06 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825348+0000", 2024-09-15T21:01:37.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2327000000000001e-05 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825385+0000", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6918999999999997e-05 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825407+0000", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2560000000000001e-05 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825508+0000", 2024-09-15T21:01:37.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000100647 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825615+0000", 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000106795 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.826443+0000", 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00082777899999999995 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044140749999999999 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334247+0000", 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50339011700000003 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334380+0000", 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013286799999999999 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:653 3.ds0 3:b220672c:::smithi02732947-15:head [rollback 16] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.648544161, 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50927399799999995, 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 653 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.560 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825450+0000", 2024-09-15T21:01:37.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3799999999999999e-06 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825459+0000", 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9959999999999999e-06 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825480+0000", 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0930000000000001e-05 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825502+0000", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.213e-05 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827235+0000", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001732691 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827493+0000", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000258139 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.828697+0000", 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012043609999999999 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002160201 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334605+0000", 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50374753100000003 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334723+0000", 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000117639 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:654 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 26.648463331999999, 2024-09-15T21:01:37.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50976337999999999, 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 654 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825531+0000", 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3030000000000001e-06 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825536+0000", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5670000000000002e-06 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825555+0000", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8680000000000001e-05 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825570+0000", 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4715999999999999e-05 2024-09-15T21:01:37.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830061+0000", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044918199999999997 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830150+0000", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8658000000000004e-05 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830574+0000", 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042412199999999998 2024-09-15T21:01:37.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.931412+0000", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100838144 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335070+0000", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40365782900000002 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335293+0000", 2024-09-15T21:01:37.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022254100000000001 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:785 3.ds0 3:b220672c:::smithi02732947-15:head [delete] snapc 1a=[1a,19,18,16] ondisk+write+known_if_redirected+full_force+supports_pool_eio e83)", 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:16.854174+0000", 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.619818511999998, 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49149152499999998, 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:37.571 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 785 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854174+0000", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854174+0000", 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854177+0000", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3499999999999999e-06 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854189+0000", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2485e-05 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854213+0000", 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3666e-05 2024-09-15T21:01:37.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.854226+0000", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3006999999999999e-05 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.954434+0000", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10020852399999999 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.954522+0000", 2024-09-15T21:01:37.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7634999999999997e-05 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.955279+0000", 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00075723399999999997 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.042667+0000", 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.087387568999999998 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:37.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.345398+0000", 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30273149199999999 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.345666+0000", 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00026756300000000001 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.576 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.57945282200000003, 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50728277399999999, 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:37.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894542+0000", 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895468+0000", 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092661499999999999 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895495+0000", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.686e-05 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.995597+0000", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100101948 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401647+0000", 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40604999000000003 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401823+0000", 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000175351 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.48875225999999999, 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.41678055400000003, 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.580 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985240+0000", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985242+0000", 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.686e-06 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985679+0000", 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043683900000000001 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.985702+0000", 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3339000000000001e-05 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085831+0000", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012874200000001 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401873+0000", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.316042347 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.402021+0000", 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000147601 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.583 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.47705513799999999, 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40530471899999998, 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996938+0000", 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.996939+0000", 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3e-06 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:37.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997370+0000", 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00043102499999999997 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997402+0000", 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2592999999999998e-05 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997419+0000", 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6670000000000001e-05 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.402088+0000", 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40466856200000001 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.402242+0000", 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015456899999999999 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:37.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:37.589 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:37.582+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:37.655 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:370: done (4 left) 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:371: read oid 38 snap -1 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:371: expect deleted 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:372: read oid 3 snap -1 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:372: expect (ObjNum 105 snap 17 seq_num 105) 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 21 current snap is 33 2024-09-15T21:01:37.660 INFO:tasks.rados.rados.0.smithi027.stdout:373: seq_num 146 ranges {1622016=49152} 2024-09-15T21:01:37.670 INFO:tasks.rados.rados.0.smithi027.stdout:373: writing smithi02732947-21 from 1622016 to 1671168 tid 1 2024-09-15T21:01:37.670 INFO:tasks.rados.rados.0.smithi027.stdout:374: read oid 30 snap -1 2024-09-15T21:01:37.670 INFO:tasks.rados.rados.0.smithi027.stdout:374: expect (ObjNum 124 snap 23 seq_num 124) 2024-09-15T21:01:37.670 INFO:tasks.rados.rados.0.smithi027.stdout:375: copy_from oid 34 from oid 35 current snap is 33 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 46 current snap is 33 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:376: seq_num 147 ranges {0=65536} 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:376: writing smithi02732947-46 from 0 to 65536 tid 1 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:377: read oid 31 snap -1 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:377: expect deleted 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 9 current snap is 33 2024-09-15T21:01:37.671 INFO:tasks.rados.rados.0.smithi027.stdout:378: seq_num 148 ranges {4194304=81920} 2024-09-15T21:01:37.695 INFO:tasks.rados.rados.0.smithi027.stdout:378: writing smithi02732947-9 from 4194304 to 4276224 tid 1 2024-09-15T21:01:37.696 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 48 current snap is 33 2024-09-15T21:01:37.696 INFO:tasks.rados.rados.0.smithi027.stdout:379: seq_num 149 ranges {0=98304} 2024-09-15T21:01:37.696 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:37.690+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:37.696 INFO:tasks.rados.rados.0.smithi027.stdout:379: writing smithi02732947-48 from 0 to 98304 tid 1 2024-09-15T21:01:37.697 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 13 current snap is 33 2024-09-15T21:01:37.697 INFO:tasks.rados.rados.0.smithi027.stdout:380: seq_num 150 ranges {4308992=114688} 2024-09-15T21:01:37.722 INFO:tasks.rados.rados.0.smithi027.stdout:380: writing smithi02732947-13 from 4308992 to 4423680 tid 1 2024-09-15T21:01:37.722 INFO:tasks.rados.rados.0.smithi027.stdout:381: delete oid 28 current snap is 33 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDPGPush(3.14s1 103/99 [PushOp(3:2f703215:::smithi02732947-30:18, version: 80'141, data_included: [0~122880], data_size: 122880, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:2f703215:::smithi02732947-30:18@80'141, size: 491520, copy_subset: [], clone_subset: {}, snapset: 22=[]:{17: [16], 18: [18], 22: [22, 21, 1f, 1b, 1a]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 491520, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.658690+0000", 2024-09-15T21:01:37.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.069117901999999995, 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.069216814000000002, 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:37.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658690+0000", 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658690+0000", 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:37.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658691+0000", 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.322e-06 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658884+0000", 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019307999999999999 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658920+0000", 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5879e-05 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.658944+0000", 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4646000000000001e-05 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.12s3 103/99 ECSubWrite(tid=173, reqid=client.4641.0:937, at_version=103'86, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:37.734 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.723414+0000", 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.0043933399999999999, 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045705160000000002, 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723414+0000", 2024-09-15T21:01:37.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723414+0000", 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723414+0000", 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:37.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723581+0000", 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016639899999999999 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723598+0000", 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.723608+0000", 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8220000000000002e-06 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:37.738 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:37.739 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 2 2024-09-15T21:01:37.739 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:37.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:37.766 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:37.759+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:37.790 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:37.783+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:37.868 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:37.869 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:37.869 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:37.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:37.927 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:37.926+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:37.931 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 1 to smithi02732947-39 2024-09-15T21:01:37.991 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:37.987+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:38.007 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.711331736999995, 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.008 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:01:38.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:01:38.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.710028825999999, 2024-09-15T21:01:38.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:01:38.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.018 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:38.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.709948901000004, 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.503465051999996, 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.027 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:01:38.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:01:38.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:01:38.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.035 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.502067607000001, 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.036 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.037 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:01:38.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.037 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:01:38.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.502028379999999, 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:01:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:01:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.300177911999995, 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.299807095999995, 2024-09-15T21:01:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:01:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.299637434000005, 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:01:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:01:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.299229244000003, 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:01:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.231949400000005, 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:01:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.200736497999998, 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:01:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:01:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.198967628000005, 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.075 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.063+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.076 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.063+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:166 3.9s0 3:9b48fc7f:::smithi02732947-22:head [copy-from ver 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.098943147, 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.71698077900000001, 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166 2024-09-15T21:01:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904687+0000", 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904696+0000", 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9899999999999992e-06 2024-09-15T21:01:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904709+0000", 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2543e-05 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904717+0000", 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904853+0000", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013647299999999999 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904974+0000", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120728 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115180+0000", 2024-09-15T21:01:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210206536 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116911+0000", 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730517 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621588+0000", 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.504676919 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621665+0000", 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7520000000000003e-05 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:466 3.9s0 3:9b48fc7f:::smithi02732947-22:head [rollback e] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.261819473999999, 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36587998700000002, 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741810+0000", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7e-06 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741817+0000", 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2099999999999996e-06 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741838+0000", 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0659e-05 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741854+0000", 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5930000000000002e-05 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802675+0000", 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060821433000000001 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802967+0000", 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029195399999999998 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804598+0000", 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016307870000000001 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809146+0000", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004548191 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107499+0000", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298352532 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107688+0000", 2024-09-15T21:01:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018959099999999999 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:467 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.261747135999997, 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36615330200000001, 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741882+0000", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3e-06 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741888+0000", 2024-09-15T21:01:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9170000000000003e-06 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741910+0000", 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1838999999999999e-05 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741928+0000", 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8016000000000002e-05 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809749+0000", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067821091 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809897+0000", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148312 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810785+0000", 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088809800000000003 2024-09-15T21:01:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.812435+0000", 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016501619999999999 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107930+0000", 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295494543 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.108034+0000", 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010402400000000001 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:506 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~540672] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.321750178999999, 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.37081094999999997, 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 506 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681880+0000", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682876+0000", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099578100000000005 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682908+0000", 2024-09-15T21:01:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2138999999999998e-05 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682923+0000", 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5577e-05 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683044+0000", 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121033 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683056+0000", 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1732999999999999e-05 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.746713+0000", 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.063656931999999999 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.747027+0000", 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031414700000000003 2024-09-15T21:01:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.749307+0000", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022794339999999999 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.841341+0000", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.092034070999999995 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052501+0000", 2024-09-15T21:01:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21116006500000001 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052689+0000", 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000187708 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:508 3.9s0 3:9b48fc7f:::smithi02732947-22:head [setxattr _header (57) in=7b] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.312689613000003, 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.459624861, 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 508 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690939+0000", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690947+0000", 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690958+0000", 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.128e-05 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690967+0000", 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691124+0000", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015739899999999999 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691145+0000", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942749+0000", 2024-09-15T21:01:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25160399300000003 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942917+0000", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167522 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.943821+0000", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090415099999999996 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.945525+0000", 2024-09-15T21:01:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017036480000000001 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150355+0000", 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20482993199999999 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150563+0000", 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020850400000000001 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:624 3.9s0 3:9b0039f5:::smithi02732947-45:head [create,setxattr [OXS[Rf]kSmSVZkcqrPoTRUsq[o]epTblNRaQggt (1858) in=40b,setxattr QrooaVmb\\N]q[ho]kgWWcr\\eocaQRtZ_`]Qsipqt (174) in=40b,setxattr Ydefbt^YQ\\dpmjectRYX^qRe[eQdQqbZhllc (770) in=36b,setxattr TV\\gUfWjiSV[PabStYoPOXcXUVsSaXdhhRnnhpfs (454) in=40b,setxattr m_O\\\\icSfcPOnoNRZ]O`X]X`ZaVOXegXNVmPglmp (734) in=40b,setxattr mf`_l`a[Y`agdjhtOpoWmnWT^dS[ScSdZWamrcjp (699) in=40b,setxattr hPm]lVO[\\[iefNWiNnaWNQr (770) in=23b,setxattr gkna_]qPX[S`[_g_ifYo]hUlgScp_jnl[^WpdheP (1614) in=40b,setxattr YSndfaQqlVQQXmZiNjPrZZa`glYedsbkkcroSttS (1333) in=40b,setxattr VVliPgssabe (770) in=11b,setxattr kdelpR\\kNn^\\r\\aeabqa\\jcQSaOXgNlVTroTaodb (1610) in=40b,setxattr a[YZZ[_nQae][Ujatlhnen_QorP_PROiQkaafPl^ (1754) in=40b,setxattr Y\\iYcXa`paqfZggTnSmaVZ[Yt`te\\rdepg^foUUX (12280) in=40b,setxattr WPicPk_V\\Z]`USR]\\Xq (770) in=19b,setxattr gqtnZjUUjhkmnQqhohYVhkliPOppPSXTshf_sVbq (877) in=40b,setxattr Nadgio[acb^P_fdps]bigQrctbcktsZdTe]Z_hhb (8628) in=40b,setxattr oUUbUt^qtWedq]`NoRbZf[jPhdNrRbjTU`m^jbpe (666) in=40b,setxattr UUhYqmbd_S[all[PkPg\\odhP\\gkshjetOj][^OWq (16140) in=40b,setxattr WSTftae_cS`q^sZiZQskrTnpRX^Tkmid^ZbYOeTo (1997) in=40b,setxattr bdcqYYhcTQq^O[TjlUb[oTXa_hoaQStVglem^VqX (981) in=40b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.366767910999997, 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50056462599999996, 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 624 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636862+0000", 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637027+0000", 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016484200000000001 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637047+0000", 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0182000000000001e-05 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637059+0000", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1892999999999999e-05 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745449+0000", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10838987999999999 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745728+0000", 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027862699999999999 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.747309+0000", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001581004 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.748624+0000", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013150099999999999 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137271+0000", 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38864758999999999 2024-09-15T21:01:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137424+0000", 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015299899999999999 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:909 3.9s0 3:9b0039f5:::smithi02732947-45:head [copy-from ver 155] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 10.382727831, 2024-09-15T21:01:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.57516277900000001, 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 909 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620903+0000", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620915+0000", 2024-09-15T21:01:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2612999999999999e-05 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620948+0000", 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2617999999999998e-05 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620965+0000", 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.715178+0000", 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094213704999999995 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.715659+0000", 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000480774 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.909645+0000", 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19398601500000001 2024-09-15T21:01:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.917783+0000", 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081374810000000002 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:28.195653+0000", 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.27787013999999999 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:28.196063+0000", 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040987800000000002 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.123 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:38.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:38.125 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:38.122+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:38.158 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.151+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.158 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 2 to smithi02732947-39 2024-09-15T21:01:38.158 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 3 to smithi02732947-39 2024-09-15T21:01:38.159 INFO:tasks.rados.rados.0.smithi027.stdout:377: done (14 left) 2024-09-15T21:01:38.159 INFO:tasks.rados.rados.0.smithi027.stdout:381: done (13 left) 2024-09-15T21:01:38.159 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 11 current snap is 33 2024-09-15T21:01:38.160 INFO:tasks.rados.rados.0.smithi027.stdout:382: seq_num 152 ranges {180224=147456} 2024-09-15T21:01:38.162 INFO:tasks.rados.rados.0.smithi027.stdout:382: writing smithi02732947-11 from 180224 to 327680 tid 1 2024-09-15T21:01:38.162 INFO:tasks.rados.rados.0.smithi027.stdout:383: setattr oid 42 current snap is 33 2024-09-15T21:01:38.164 INFO:tasks.rados.rados.0.smithi027.stdout:384: read oid 2 snap -1 2024-09-15T21:01:38.164 INFO:tasks.rados.rados.0.smithi027.stdout:384: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-15T21:01:38.164 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:01:38.228 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.223+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:38.261 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 4 to smithi02732947-39 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s1 104/96 ECSubWrite(tid=186, reqid=osd.11.0:186, at_version=104'71, trim_to=0'0, roll_forward_to=103'69))", 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.160324+0000", 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.10024714699999999, 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100298185, 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.160324+0000", 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.160324+0000", 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.160325+0000", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8400000000000003e-07 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.160351+0000", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6338000000000001e-05 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.160370+0000", 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9035999999999999e-05 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260462+0000", 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.1000916 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260634+0000", 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017208600000000001 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s1 104/96 ECSubWrite(tid=187, reqid=osd.11.0:187, at_version=104'73, trim_to=0'0, roll_forward_to=103'69))", 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.260532+0000", 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9774999999999998e-05, 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021930800000000001, 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260532+0000", 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260532+0000", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260533+0000", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1230000000000001e-06 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260571+0000", 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8192000000000003e-05 2024-09-15T21:01:38.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260591+0000", 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9746e-05 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.260603+0000", 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2176e-05 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.268 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:38.269 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 2 2024-09-15T21:01:38.269 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:38.272 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.267+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:38.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:38.278 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.271+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:38.278 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.271+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6 to read 0,1,4,5 2024-09-15T21:01:38.289 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.283+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:38.289 INFO:tasks.rados.rados.0.smithi027.stdout:380: finishing write tid 1 to smithi02732947-13 2024-09-15T21:01:38.289 INFO:tasks.rados.rados.0.smithi027.stdout:380: finishing write tid 2 to smithi02732947-13 2024-09-15T21:01:38.290 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.283+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6 to read 0,1,4,5 2024-09-15T21:01:38.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.287+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.293 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.287+0000 7fde75110640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:38.303 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.302+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.306 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.299+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,4,5 2024-09-15T21:01:38.307 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.303+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6 to read 0,1,4,5 2024-09-15T21:01:38.328 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:38.323+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:38.364 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.359+0000 7fde75911640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:38.365 INFO:tasks.rados.rados.0.smithi027.stdout:379: finishing write tid 1 to smithi02732947-48 2024-09-15T21:01:38.365 INFO:tasks.rados.rados.0.smithi027.stdout:379: finishing write tid 2 to smithi02732947-48 2024-09-15T21:01:38.370 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.363+0000 7fde79118640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,4 to read 0,1,4,5 2024-09-15T21:01:38.377 INFO:tasks.rados.rados.0.smithi027.stdout:376: finishing write tid 1 to smithi02732947-46 2024-09-15T21:01:38.379 INFO:tasks.rados.rados.0.smithi027.stdout:374: expect (ObjNum 124 snap 23 seq_num 124) 2024-09-15T21:01:38.379 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:38.392 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:38.390+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:38.392 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:38.390+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:38.407 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:38.408 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:38.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:38.429 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.422+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:38.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:38.446 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:38.459 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.458+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:38.459 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.458+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:01:38.459 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.458+0000 7f84bc50d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5,7 to read 0,1,4,5 2024-09-15T21:01:38.468 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:38.466+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,7 to read 0,1,4,5 2024-09-15T21:01:38.486 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:38.491 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:38.490+0000 7f8652952640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,6,7 to read 0,1,4,5 2024-09-15T21:01:38.514 INFO:tasks.rados.rados.0.smithi027.stdout:378: finishing write tid 1 to smithi02732947-9 2024-09-15T21:01:38.514 INFO:tasks.rados.rados.0.smithi027.stdout:378: finishing write tid 2 to smithi02732947-9 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:378: finishing write tid 3 to smithi02732947-9 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:378: oid 9 updating version 0 to 98 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:378: oid 9 version 98 is already newer than 97 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 9 v 98 (ObjNum 148 snap 33 seq_num 148) dirty exists 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:378: left oid 9 (ObjNum 148 snap 33 seq_num 148) 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:379: finishing write tid 3 to smithi02732947-48 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:379: oid 48 updating version 0 to 21 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:379: oid 48 updating version 21 to 22 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 48 v 22 (ObjNum 149 snap 33 seq_num 149) dirty exists 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:379: left oid 48 (ObjNum 149 snap 33 seq_num 149) 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:376: finishing write tid 2 to smithi02732947-46 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:374: done (15 left) 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:378: done (14 left) 2024-09-15T21:01:38.515 INFO:tasks.rados.rados.0.smithi027.stdout:379: done (13 left) 2024-09-15T21:01:38.516 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 29 current snap is 33 2024-09-15T21:01:38.516 INFO:tasks.rados.rados.0.smithi027.stdout:385: seq_num 153 ranges {3260416=163840} 2024-09-15T21:01:38.529 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:38.526+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:38.541 INFO:tasks.rados.rados.0.smithi027.stdout:385: writing smithi02732947-29 from 3260416 to 3424256 tid 1 2024-09-15T21:01:38.541 INFO:tasks.rados.rados.0.smithi027.stdout:386: rollback oid 17 current snap is 33 2024-09-15T21:01:38.541 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 17 to 32 2024-09-15T21:01:38.541 INFO:tasks.rados.rados.0.smithi027.stdout:387: setattr oid 43 current snap is 33 2024-09-15T21:01:38.542 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 16 2024-09-15T21:01:38.575 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.567+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:38.592 INFO:tasks.rados.rados.0.smithi027.stdout:380: finishing write tid 3 to smithi02732947-13 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:380: oid 13 updating version 0 to 186 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:380: oid 13 version 186 is already newer than 185 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 13 v 186 (ObjNum 150 snap 33 seq_num 150) dirty exists 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:380: left oid 13 (ObjNum 150 snap 33 seq_num 150) 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:380: done (15 left) 2024-09-15T21:01:38.593 INFO:tasks.rados.rados.0.smithi027.stdout:388: snap_create 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.219246322000004, 2024-09-15T21:01:38.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:01:38.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.828850430000003, 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30705592100000001, 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763551+0000", 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763784+0000", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023310900000000001 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763818+0000", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3855e-05 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763835+0000", 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6393e-05 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964212+0000", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200376947 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964244+0000", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2035000000000001e-05 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070501+0000", 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10625775599999999 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070605+0000", 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103706 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=5, reqid=client.4641.0:430, at_version=59'75, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.823576101, 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.402708554, 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768826+0000", 2024-09-15T21:01:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000002e-06 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769062+0000", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581599999999999 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769156+0000", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4553000000000001e-05 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769216+0000", 2024-09-15T21:01:38.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9932000000000001e-05 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966630+0000", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19741336600000001 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966643+0000", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3609e-05 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171495+0000", 2024-09-15T21:01:38.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20485210600000001 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171532+0000", 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6711999999999997e-05 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.720606056000001, 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299792744, 2024-09-15T21:01:38.609 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871820+0000", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5873000000000001e-05 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871847+0000", 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6792e-05 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871873+0000", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6075999999999999e-05 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068219+0000", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19634641799999999 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068236+0000", 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6912999999999999e-05 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171560+0000", 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10332366 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171586+0000", 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6438999999999999e-05 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.720350912000001, 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29954117000000002, 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872055+0000", 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8370000000000002e-06 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872067+0000", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493e-05 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872079+0000", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1456e-05 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171223+0000", 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299144415 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171590+0000", 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036652199999999998 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=57, reqid=client.4641.0:465, at_version=60'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.79159568, 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30393555799999999, 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:38.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-07 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800860+0000", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5575000000000002e-05 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800883+0000", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3116e-05 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800903+0000", 2024-09-15T21:01:38.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9426e-05 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001512+0000", 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20060972299999999 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001540+0000", 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7674999999999998e-05 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104682+0000", 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103142055 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104739+0000", 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7481000000000001e-05 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=58, reqid=client.4641.0:466, at_version=60'38, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.787732556000002, 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30238109299999999, 2024-09-15T21:01:38.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804669+0000", 2024-09-15T21:01:38.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7969999999999999e-06 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804721+0000", 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1820999999999997e-05 2024-09-15T21:01:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804762+0000", 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1115999999999997e-05 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804786+0000", 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4352e-05 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103533+0000", 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29874738099999998 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103546+0000", 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2768e-05 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.106961+0000", 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415094 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107048+0000", 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6763999999999998e-05 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=59, reqid=client.4641.0:467, at_version=60'39, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.781579370000003, 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29628899199999997, 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810822+0000", 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0430000000000002e-06 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810872+0000", 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9962e-05 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810913+0000", 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0642000000000002e-05 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810940+0000", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6662e-05 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104296+0000", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29335650600000002 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104312+0000", 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5559000000000001e-05 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107057+0000", 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027455829999999998 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107109+0000", 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2034999999999999e-05 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:38.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.692428601000003, 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20751788099999999, 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899973+0000", 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900092+0000", 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118466 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900123+0000", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1149000000000003e-05 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900146+0000", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779000000000001e-05 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900331+0000", 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000184594 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900362+0000", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0559000000000003e-05 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107409+0000", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20704719299999999 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107489+0000", 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0061000000000005e-05 2024-09-15T21:01:38.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=16, reqid=client.4641.0:462, at_version=60'83, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.692203751999998, 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20735974700000001, 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900197+0000", 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900419+0000", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022236900000000001 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900447+0000", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7373e-05 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900472+0000", 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981999999999999e-05 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002446+0000", 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101974907 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002464+0000", 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045999999999999e-05 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107496+0000", 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105032797 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107556+0000", 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259000000000003e-05 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=17, reqid=client.4641.0:463, at_version=60'84, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:38.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.691881449, 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20715383400000001, 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:38.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900520+0000", 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900569+0000", 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9608999999999999e-05 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900593+0000", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4034999999999999e-05 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900613+0000", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103732+0000", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.203119718 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103749+0000", 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107564+0000", 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038144160000000002 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107672+0000", 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108294 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 73/72 ECSubWrite(tid=91, reqid=osd.8.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=73'55))", 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.216979481999999, 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29881405799999999, 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:38.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375421+0000", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375434+0000", 2024-09-15T21:01:38.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2388000000000001e-05 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375447+0000", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2979000000000001e-05 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375456+0000", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8870000000000004e-06 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572431+0000", 2024-09-15T21:01:38.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19697500200000001 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572450+0000", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8260000000000001e-05 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674174+0000", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10172400700000001 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674234+0000", 2024-09-15T21:01:38.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0445000000000001e-05 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.0s1 87/84 ECSubWrite(tid=135, reqid=client.4641.0:844, at_version=87'40, trim_to=0'0, roll_forward_to=86'38))", 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.617437606999999, 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.204339415, 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974962+0000", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974964+0000", 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.367e-06 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975181+0000", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021703299999999999 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975220+0000", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.82e-05 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.975244+0000", 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4528999999999998e-05 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.175627+0000", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20038291699999999 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.175660+0000", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3334999999999998e-05 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.179198+0000", 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0035375179999999999 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.179301+0000", 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103516 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 87/28 ECSubWrite(tid=355, reqid=client.4641.0:821, at_version=87'78, trim_to=0'0, roll_forward_to=86'75))", 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.247464232999999, 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20320760800000001, 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344935+0000", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344937+0000", 2024-09-15T21:01:38.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1430000000000001e-06 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.344980+0000", 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2660999999999997e-05 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345013+0000", 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3047999999999999e-05 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445166+0000", 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100153248 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445361+0000", 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019436099999999999 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.445378+0000", 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7278999999999998e-05 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.548055+0000", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102676964 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.548143+0000", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7904000000000004e-05 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=398, reqid=osd.5.0:398, at_version=104'95, trim_to=0'0, roll_forward_to=103'91))", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:38.657 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.90885248299999999, 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.303411086, 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:38.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683549+0000", 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7129999999999999e-06 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683602+0000", 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3097999999999999e-05 2024-09-15T21:01:38.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683643+0000", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1372000000000003e-05 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683666+0000", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3079e-05 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.684664+0000", 2024-09-15T21:01:38.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099762700000000006 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.684675+0000", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1342000000000001e-05 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.986889+0000", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30221355700000002 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.986958+0000", 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9298e-05 2024-09-15T21:01:38.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=104'183, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.90636065700000001, 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50142837399999995, 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686063+0000", 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3509000000000001e-05 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686076+0000", 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3621999999999999e-05 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.786146+0000", 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100069362 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.885328+0000", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099182114000000002 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.885344+0000", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5911999999999999e-05 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187436+0000", 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.302092527 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187467+0000", 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0930999999999999e-05 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=400, reqid=client.4641.0:968, at_version=104'97, trim_to=0'0, roll_forward_to=103'91))", 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.79856403399999998, 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39368508299999999, 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793837+0000", 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.034e-06 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793943+0000", 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106263 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793957+0000", 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4535000000000001e-05 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793967+0000", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2690000000000005e-06 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985418+0000", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19145102 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985435+0000", 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187484+0000", 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20204925800000001 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187521+0000", 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6532000000000002e-05 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=129, reqid=client.4641.0:974, at_version=104'185, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.76701903599999999, 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36217170399999998, 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825383+0000", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7099999999999999e-06 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825437+0000", 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3446999999999998e-05 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825458+0000", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1579000000000001e-05 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825474+0000", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5277999999999998e-05 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985924+0000", 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16045072099999999 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985940+0000", 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5705999999999998e-05 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187525+0000", 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20158479400000001 2024-09-15T21:01:38.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187552+0000", 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7469000000000001e-05 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=130, reqid=client.4641.0:975, at_version=104'186, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.76225482300000003, 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.35784861000000001, 2024-09-15T21:01:38.675 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830146+0000", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.217e-06 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830176+0000", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0188e-05 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830190+0000", 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4341e-05 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830198+0000", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3379999999999999e-06 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187198+0000", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.35700012599999997 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187207+0000", 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187953+0000", 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074618899999999997 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187993+0000", 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0111999999999998e-05 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=404, reqid=client.4641.0:969, at_version=104'98, trim_to=0'0, roll_forward_to=104'93))", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.688067451, 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28325234199999999, 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904333+0000", 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8800000000000002e-07 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904350+0000", 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6988999999999999e-05 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904362+0000", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2149e-05 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904374+0000", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1827999999999999e-05 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.988290+0000", 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.083916404 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.988317+0000", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187556+0000", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19923970299999999 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187584+0000", 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8136e-05 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.684 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:38.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.658+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.658+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.658+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,5 to read 0,1,4,5 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.658+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.662+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.686 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.662+0000 7f84bb50b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2 to read 0,1,4,5 2024-09-15T21:01:38.687 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.678+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4 to read 0,1,4,5 2024-09-15T21:01:38.687 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:38.682+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:38.689 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:38.683+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:38.690 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.682+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:38.692 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 5 to smithi02732947-39 2024-09-15T21:01:38.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:38.766 INFO:tasks.rados.rados.0.smithi027.stdout:382: finishing write tid 1 to smithi02732947-11 2024-09-15T21:01:38.766 INFO:tasks.rados.rados.0.smithi027.stdout:382: finishing write tid 2 to smithi02732947-11 2024-09-15T21:01:38.769 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:38.762+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:38.769 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:38.762+0000 7fe63d23f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3 to read 0,1,4,5 2024-09-15T21:01:38.784 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.782+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1 to read 0,1,4,5 2024-09-15T21:01:38.784 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:38.782+0000 7f84bf513640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2 to read 0,1,4,5 2024-09-15T21:01:38.786 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:38.779+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:38.807 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:38.807 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:38.831 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:38.831 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:38.848 INFO:tasks.rados.rados.0.smithi027.stdout:375: finishing copy_from to smithi02732947-34 2024-09-15T21:01:38.848 INFO:tasks.rados.rados.0.smithi027.stdout:375: got expected ENOENT (src dne) 2024-09-15T21:01:38.848 INFO:tasks.rados.rados.0.smithi027.stdout:375: finishing copy_from racing read to smithi02732947-34 2024-09-15T21:01:38.855 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:38.861 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.6s5 105/96 ECSubRead(tid=184, to_read={3:6097bd07:::smithi02732947-11:head=0,4096,0}, subchunks={3:6097bd07:::smithi02732947-11:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.768128+0000", 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.092549662000000005, 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.092593206999999997, 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.768128+0000", 2024-09-15T21:01:38.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.768128+0000", 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.768129+0000", 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.514e-06 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.768147+0000", 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7699000000000001e-05 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.768180+0000", 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3345e-05 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 104/96 ECSubWrite(tid=412, reqid=client.4641.0:989, at_version=104'232, trim_to=0'0, roll_forward_to=104'227))", 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.775270+0000", 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.085407355000000004, 2024-09-15T21:01:38.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085520864000000002, 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775270+0000", 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775270+0000", 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775270+0000", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775289+0000", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8297000000000001e-05 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775299+0000", 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0165e-05 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.775307+0000", 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2579999999999995e-06 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.5s7 105/96 ECSubWrite(tid=413, reqid=client.4641.0:990, at_version=105'233, trim_to=0'0, roll_forward_to=104'227))", 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.782900+0000", 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.077777052999999999, 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.077960565999999995, 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:38.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782900+0000", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782900+0000", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782902+0000", 2024-09-15T21:01:38.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9199999999999998e-06 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782936+0000", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3748999999999998e-05 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782966+0000", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0175000000000002e-05 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.782989+0000", 2024-09-15T21:01:38.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2900000000000001e-05 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 3 2024-09-15T21:01:38.870 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:38.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:38.880 INFO:tasks.rados.rados.0.smithi027.stdout:384: expect (ObjNum 135 snap 28 seq_num 135) 2024-09-15T21:01:38.880 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:38.887 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:38.888 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:38.888 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:38.883+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:38.892 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.886+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:01:38.892 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.886+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:01:38.892 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.886+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:01:38.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:38.922+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6 to read 0,1,4,5 2024-09-15T21:01:38.930 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:38.922+0000 7fe639237640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,6,7 to read 0,1,4,5 2024-09-15T21:01:38.988 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:38.982+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:39.003 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:39.003 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:39.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:39.025 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1167597 2024-09-15T21:01:39.038 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:39.031+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:39.089 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:39.086+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:39.107 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:39.108 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:39.112 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 2712547 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:430 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~770048] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.531813427000003, 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67855575999999995, 2024-09-15T21:01:39.146 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 430 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610260+0000", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611394+0000", 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011336479999999999 2024-09-15T21:01:39.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611427+0000", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3259000000000001e-05 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611443+0000", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-05 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.765717+0000", 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15427427099999999 2024-09-15T21:01:39.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.766028+0000", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031123899999999999 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768367+0000", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023391079999999999 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858789+0000", 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090422016999999993 2024-09-15T21:01:39.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288681+0000", 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42989205899999999 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288813+0000", 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131722 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:431 3.4s0 3:22a20f07:::smithi02732947-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.530599543999998, 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.77799016399999998, 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.152 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 431 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611472+0000", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.176e-06 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611480+0000", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2269999999999996e-06 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611491+0000", 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1776e-05 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611500+0000", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1929999999999993e-06 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770200+0000", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15870036200000001 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770513+0000", 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031272599999999999 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.771421+0000", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090811600000000004 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858791+0000", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087370619999999996 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389309+0000", 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.53051743900000004 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389461+0000", 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015252899999999999 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:432 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 48.530557672999997, 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78326424900000002, 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 432 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611514+0000", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.004e-06 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611517+0000", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4029999999999999e-06 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611528+0000", 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611536+0000", 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772246+0000", 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.160710251 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772520+0000", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027356700000000001 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390423+0000", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61790300099999995 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390689+0000", 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026639399999999998 2024-09-15T21:01:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.394777+0000", 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040880530000000003 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:576 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~786432] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:39.162 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.536144428999997, 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61536784200000005, 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 576 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:39.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605928+0000", 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607006+0000", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001077862 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607032+0000", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5885999999999999e-05 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607045+0000", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662488+0000", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055442577999999999 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662791+0000", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030328799999999997 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.665971+0000", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031791979999999998 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816856+0000", 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15088531299999999 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221142+0000", 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40428599700000001 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221294+0000", 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152297 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.167 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:577 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~376832] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.532837667000003, 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61247240599999997, 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 577 2024-09-15T21:01:39.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609236+0000", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6460000000000002e-06 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610212+0000", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097600200000000001 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610242+0000", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9963000000000001e-05 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610268+0000", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712231+0000", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101962937 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712411+0000", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018002200000000001 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.714107+0000", 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696896 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816859+0000", 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102751021 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221545+0000", 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40468618099999998 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221705+0000", 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016079899999999999 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:578 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.531761377999999, 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61180690699999996, 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 578 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:39.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610311+0000", 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610321+0000", 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9690000000000006e-06 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610342+0000", 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0375999999999999e-05 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610357+0000", 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5237e-05 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817100+0000", 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20674321900000001 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817311+0000", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021106099999999999 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818686+0000", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013748009999999999 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.919851+0000", 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101164695 2024-09-15T21:01:39.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221947+0000", 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30209587999999998 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.222116+0000", 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016968899999999999 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:774 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~573440] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.484716233, 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.79679720700000001, 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.179 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:39.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657358+0000", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0939999999999999e-06 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658209+0000", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085119500000000001 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658243+0000", 2024-09-15T21:01:39.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3707000000000001e-05 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658259+0000", 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6985000000000001e-05 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837273+0000", 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17901335199999999 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837446+0000", 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017337400000000001 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.840435+0000", 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029892199999999999 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853927+0000", 2024-09-15T21:01:39.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013491981 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.453952+0000", 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60002478699999995 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.454152+0000", 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199512 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:775 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.48131557, 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89396708599999997, 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.185 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660758+0000", 2024-09-15T21:01:39.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661604+0000", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084594499999999997 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661636+0000", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2280000000000003e-05 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661651+0000", 2024-09-15T21:01:39.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4962e-05 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844564+0000", 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18291332699999999 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844791+0000", 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000226646 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.846827+0000", 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020363400000000002 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853934+0000", 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071071449999999996 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554542+0000", 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70060748500000003 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554722+0000", 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018044600000000001 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:776 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.477833477000001, 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89087423499999996, 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664240+0000", 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4930000000000001e-06 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665080+0000", 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084009500000000001 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665112+0000", 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1961000000000001e-05 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665127+0000", 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5228999999999999e-05 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848176+0000", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18304937199999999 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848475+0000", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029845799999999998 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.850672+0000", 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021974080000000001 2024-09-15T21:01:39.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853936+0000", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032637159999999998 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554980+0000", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70104405299999994 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555112+0000", 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013145 2024-09-15T21:01:39.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:777 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~311296] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.47605519, 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88944256600000005, 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.196 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 777 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666018+0000", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159999999999999e-06 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666502+0000", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048393600000000002 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666534+0000", 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2255000000000002e-05 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766637+0000", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100103422 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851711+0000", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085073812999999998 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851901+0000", 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018974899999999999 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853195+0000", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012942540000000001 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.854601+0000", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001405445 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555319+0000", 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70071828300000005 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555458+0000", 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013909299999999999 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:778 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.375396563999999, 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88938152199999998, 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.201 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:39.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766676+0000", 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9400000000000001e-06 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766686+0000", 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766707+0000", 2024-09-15T21:01:39.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:39.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866809+0000", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100102237 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866970+0000", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160848 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867191+0000", 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220576 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867924+0000", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073345699999999997 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868858+0000", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093380199999999996 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.655861+0000", 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78700323500000002 2024-09-15T21:01:39.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.656056+0000", 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019429899999999999 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:779 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.275217366, 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99582967, 2024-09-15T21:01:39.207 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866856+0000", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.463e-06 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866868+0000", 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2398999999999999e-05 2024-09-15T21:01:39.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866896+0000", 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7370999999999999e-05 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866914+0000", 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8788999999999999e-05 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868425+0000", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001510174 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868575+0000", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150109 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657374+0000", 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78879953300000005 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657536+0000", 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016209800000000001 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.862683+0000", 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.205146734 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:783 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~4751360,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.50410514, 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.636081864, 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 783 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637967+0000", 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2899999999999999e-06 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637973+0000", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-06 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637991+0000", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7825e-05 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.638004+0000", 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3213e-05 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039573+0000", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40156882999999999 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039782+0000", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020919100000000001 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.274047+0000", 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.234265375 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:838 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~737280] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.415469783999999, 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61883177, 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 838 2024-09-15T21:01:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726605+0000", 2024-09-15T21:01:39.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7730000000000001e-06 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727781+0000", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011759159999999999 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727819+0000", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8733999999999999e-05 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727838+0000", 2024-09-15T21:01:39.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8811999999999999e-05 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728035+0000", 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019682099999999999 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728056+0000", 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0971999999999998e-05 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.731965+0000", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039089399999999996 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732120+0000", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015466000000000001 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.734867+0000", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027471489999999999 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946633+0000", 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.211766446 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345294+0000", 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39866105299999999 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345433+0000", 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013849399999999999 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:839 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~770048] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.409869785000001, 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61379308200000005, 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 839 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732204+0000", 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-15T21:01:39.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733438+0000", 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001234392 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733471+0000", 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3414999999999998e-05 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733491+0000", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9698999999999999e-05 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837640+0000", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104148692 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837807+0000", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167195 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.840395+0000", 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025880370000000001 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946894+0000", 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106498578 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345806+0000", 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39891273999999999 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345994+0000", 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018767799999999999 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:840 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~540672] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.406277238000001, 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61059644300000004, 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 840 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:39.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735797+0000", 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.67e-06 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736891+0000", 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010939680000000001 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736923+0000", 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1653999999999999e-05 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736939+0000", 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6045999999999998e-05 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943125+0000", 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20618578000000001 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943433+0000", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030832599999999999 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.945306+0000", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018727699999999999 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.047848+0000", 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102542726 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346245+0000", 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29839692899999998 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346390+0000", 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014457399999999999 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:841 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.233 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.404361211000001, 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60917734099999998, 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 841 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737713+0000", 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0230000000000001e-06 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738201+0000", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048861200000000003 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738252+0000", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0773000000000003e-05 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738276+0000", 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3825e-05 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.148832+0000", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41055591200000002 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.149003+0000", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017122200000000001 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.150384+0000", 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013808959999999999 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154322+0000", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039383689999999997 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346706+0000", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.192384097 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346887+0000", 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180612 2024-09-15T21:01:39.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:842 3.4s0 3:261a50f9:::smithi02732947-12:head [setxattr _header (57) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.403754008, 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70825646600000003, 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 842 2024-09-15T21:01:39.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738318+0000", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738328+0000", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7599999999999997e-06 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738350+0000", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1471000000000002e-05 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738394+0000", 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4484000000000003e-05 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151567+0000", 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.413173238 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151818+0000", 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025079399999999998 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153019+0000", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012006740000000001 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154654+0000", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00163489 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446326+0000", 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29167209999999999 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446573+0000", 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000247328 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:843 3.4s0 3:261a50f9:::smithi02732947-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.403646384999998, 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.91369339800000005, 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.244 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 843 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:39.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738426+0000", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4169999999999999e-06 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738435+0000", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0100000000000001e-06 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738461+0000", 2024-09-15T21:01:39.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6605e-05 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738477+0000", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5364999999999999e-05 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153751+0000", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41527431799999998 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153990+0000", 2024-09-15T21:01:39.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000238698 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447565+0000", 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29357503600000001 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447832+0000", 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026724700000000002 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.652118+0000", 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20428570200000001 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:976 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.419631785, 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.86952956000000003, 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:39.249 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 976 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:39.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722440+0000", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0499999999999999e-06 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722444+0000", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1030000000000004e-06 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722458+0000", 2024-09-15T21:01:39.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3712e-05 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722466+0000", 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1440000000000003e-06 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830570+0000", 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10810450000000001 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:39.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830715+0000", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014405 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.389970+0000", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.55925579599999997 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.390085+0000", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114455 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:39.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.591969+0000", 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20188375 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:39.254 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:39.255 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:39.256 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:39.187+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:39.187+0000 7fde78116640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:39.256 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:39.187+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5 to read 0,1,4,5 2024-09-15T21:01:39.256 INFO:tasks.rados.rados.0.smithi027.stdout:382: finishing write tid 3 to smithi02732947-11 2024-09-15T21:01:39.256 INFO:tasks.rados.rados.0.smithi027.stdout:382: oid 11 updating version 0 to 77 2024-09-15T21:01:39.256 INFO:tasks.rados.rados.0.smithi027.stdout:382: oid 11 version 77 is already newer than 76 2024-09-15T21:01:39.256 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 11 v 77 (ObjNum 152 snap 33 seq_num 152) dirty exists 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:382: left oid 11 (ObjNum 152 snap 33 seq_num 152) 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 42 v 229 (ObjNum 132 snap 26 seq_num 132) dirty exists 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:386: finishing rollback tid 0 to smithi02732947-17 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:376: finishing write tid 3 to smithi02732947-46 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:376: oid 46 updating version 0 to 133 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:376: oid 46 version 133 is already newer than 132 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 46 v 133 (ObjNum 147 snap 33 seq_num 147) dirty exists 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:376: left oid 46 (ObjNum 147 snap 33 seq_num 147) 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:373: finishing write tid 1 to smithi02732947-21 2024-09-15T21:01:39.257 INFO:tasks.rados.rados.0.smithi027.stdout:373: finishing write tid 2 to smithi02732947-21 2024-09-15T21:01:39.257 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:39.258 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:39.258 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:39.250+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:39.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:39.264 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:39.264 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:39.274 INFO:tasks.rados.rados.0.smithi027.stdout:385: finishing write tid 1 to smithi02732947-29 2024-09-15T21:01:39.274 INFO:tasks.rados.rados.0.smithi027.stdout:385: finishing write tid 2 to smithi02732947-29 2024-09-15T21:01:39.274 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.266+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:39.274 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.266+0000 7fe63ba3c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:39.277 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 6 to smithi02732947-39 2024-09-15T21:01:39.294 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:39.287+0000 7fde7410e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,5,7 to read 0,1,4,5 2024-09-15T21:01:39.306 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:39.306 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:39.336 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:39.356 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:39.354+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:39.359 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.350+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5 to read 0,1,4,5 2024-09-15T21:01:39.360 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.354+0000 7fe63fa44640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,5,7 to read 0,1,4,5 2024-09-15T21:01:39.366 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:39.366 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:39.376 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.370+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:39.399 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.390+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:39.432 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:39.433 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:39.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:39.456 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:39.451+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:39.457 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:39.450+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:39.476 INFO:tasks.rados.rados.0.smithi027.stdout:373: finishing write tid 3 to smithi02732947-21 2024-09-15T21:01:39.476 INFO:tasks.rados.rados.0.smithi027.stdout:373: oid 21 updating version 0 to 76 2024-09-15T21:01:39.476 INFO:tasks.rados.rados.0.smithi027.stdout:373: oid 21 updating version 76 to 77 2024-09-15T21:01:39.476 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 21 v 77 (ObjNum 146 snap 33 seq_num 146) dirty exists 2024-09-15T21:01:39.476 INFO:tasks.rados.rados.0.smithi027.stdout:373: left oid 21 (ObjNum 146 snap 33 seq_num 146) 2024-09-15T21:01:39.477 INFO:tasks.rados.rados.0.smithi027.stdout:386: finishing rollback tid 1 to smithi02732947-17 2024-09-15T21:01:39.477 INFO:tasks.rados.rados.0.smithi027.stdout:386: finishing rollback tid 2 to smithi02732947-17 2024-09-15T21:01:39.477 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 17 v 233 (ObjNum 92 snap 14 seq_num 92) dirty exists 2024-09-15T21:01:39.478 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 45 v 156 (ObjNum 116 snap 22 seq_num 116) dirty exists 2024-09-15T21:01:39.491 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:39.487+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:39.501 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:39.494+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:364: done (15 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:371: done (14 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:373: done (13 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:375: done (12 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:376: done (11 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:382: done (10 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:383: done (9 left) 2024-09-15T21:01:39.554 INFO:tasks.rados.rados.0.smithi027.stdout:384: done (8 left) 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:386: done (7 left) 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:388: done (6 left) 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:389: read oid 8 snap -1 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:389: expect (ObjNum 141 snap 32 seq_num 141) 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 25 current snap is 34 2024-09-15T21:01:39.555 INFO:tasks.rados.rados.0.smithi027.stdout:390: seq_num 154 ranges {4751360=180224} 2024-09-15T21:01:39.558 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:39.550+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:39.617 INFO:tasks.rados.rados.0.smithi027.stdout:390: writing smithi02732947-25 from 4751360 to 4931584 tid 1 2024-09-15T21:01:39.617 INFO:tasks.rados.rados.0.smithi027.stdout:391: delete oid 22 current snap is 34 2024-09-15T21:01:39.617 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:39.618 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:39.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:39.660 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:39.654+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:39.664 INFO:tasks.rados.rados.0.smithi027.stdout:385: finishing write tid 3 to smithi02732947-29 2024-09-15T21:01:39.664 INFO:tasks.rados.rados.0.smithi027.stdout:385: oid 29 updating version 0 to 87 2024-09-15T21:01:39.664 INFO:tasks.rados.rados.0.smithi027.stdout:385: oid 29 version 87 is already newer than 86 2024-09-15T21:01:39.664 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 29 v 87 (ObjNum 153 snap 33 seq_num 153) dirty exists 2024-09-15T21:01:39.664 INFO:tasks.rados.rados.0.smithi027.stdout:385: left oid 29 (ObjNum 153 snap 33 seq_num 153) 2024-09-15T21:01:39.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:39.668 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:39.668 INFO:tasks.thrashosds.thrasher:inject_pause on osd.8 2024-09-15T21:01:39.669 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:01:39.669 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:01:39.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok config set filestore_inject_stall 3 2024-09-15T21:01:39.683 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:01:39.756 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:39.756 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:39.760 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:39.755+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:39.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:39.790 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 43 v 19 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T21:01:39.798 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:39.798 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:01:39.798 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:39.861 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:39.858+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:39.897 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:39.897 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:39.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:39.951 INFO:tasks.rados.rados.0.smithi027.stdout:390: finishing write tid 1 to smithi02732947-25 2024-09-15T21:01:39.952 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:39.950+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:39.953 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:39.950+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:39.954 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:39.950+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5 to read 0,1,4,5 2024-09-15T21:01:39.954 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:39.950+0000 7f5a779c2640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,5,6 to read 0,1,4,5 2024-09-15T21:01:39.961 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:39.958+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:40.017 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.010+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:40.018 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.010+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: finishing write tid 7 to smithi02732947-39 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 updating version 0 to 133 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 updating version 133 to 137 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 version 137 is already newer than 134 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 version 137 is already newer than 136 2024-09-15T21:01:40.019 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 version 137 is already newer than 132 2024-09-15T21:01:40.020 INFO:tasks.rados.rados.0.smithi027.stdout:365: oid 39 version 137 is already newer than 135 2024-09-15T21:01:40.020 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 39 v 137 (ObjNum 143 snap 33 seq_num 143) dirty exists 2024-09-15T21:01:40.020 INFO:tasks.rados.rados.0.smithi027.stdout:365: left oid 39 (ObjNum 143 snap 33 seq_num 143) 2024-09-15T21:01:40.045 INFO:tasks.rados.rados.0.smithi027.stdout:390: finishing write tid 2 to smithi02732947-25 2024-09-15T21:01:40.063 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:40.062+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:40.085 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.085 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:40.121 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.114+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:40.121 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.114+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:40.121 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.114+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:01:40.127 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.118+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:40.127 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.118+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4 to read 0,1,4,5 2024-09-15T21:01:40.127 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.122+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5 to read 0,1,4,5 2024-09-15T21:01:40.163 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:40.162+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:40.231 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.232 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:40.248 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:40.243+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:40.248 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:40.243+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,7 to read 0,1,4,5 2024-09-15T21:01:40.248 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:40.243+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,7 to read 0,1,4,5 2024-09-15T21:01:40.249 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:40.243+0000 7fde78917640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,5,7 to read 0,1,4,5 2024-09-15T21:01:40.290 INFO:tasks.rados.rados.0.smithi027.stdout:366: finishing copy_from racing read to smithi02732947-50 2024-09-15T21:01:40.290 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.282+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:40.331 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.326+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:40.332 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.326+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5 to read 0,1,4,5 2024-09-15T21:01:40.332 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:40.326+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,5,6 to read 0,1,4,5 2024-09-15T21:01:40.335 INFO:tasks.rados.rados.0.smithi027.stdout:372: expect (ObjNum 105 snap 17 seq_num 105) 2024-09-15T21:01:40.335 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:40.361 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:40.364 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:40.362+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:40.393 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 996834 2024-09-15T21:01:40.402 INFO:tasks.rados.rados.0.smithi027.stdout:390: finishing write tid 3 to smithi02732947-25 2024-09-15T21:01:40.402 INFO:tasks.rados.rados.0.smithi027.stdout:390: oid 25 updating version 0 to 236 2024-09-15T21:01:40.402 INFO:tasks.rados.rados.0.smithi027.stdout:390: oid 25 version 236 is already newer than 235 2024-09-15T21:01:40.402 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 25 v 236 (ObjNum 154 snap 34 seq_num 154) dirty exists 2024-09-15T21:01:40.403 INFO:tasks.rados.rados.0.smithi027.stdout:390: left oid 25 (ObjNum 154 snap 34 seq_num 154) 2024-09-15T21:01:40.422 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.422 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:40.465 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:40.459+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:40.490 INFO:tasks.rados.rados.0.smithi027.stdout:366: finishing copy_from to smithi02732947-50 2024-09-15T21:01:40.491 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 50 v 67 (ObjNum 121 snap 23 seq_num 121) dirty exists 2024-09-15T21:01:40.550 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is set 2024-09-15T21:01:40.566 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:40.562+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:40.611 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.611 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.625 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:40.667 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:40.662+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:40.768 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:40.763+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:40.802 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.803 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:40.869 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:40.863+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:40.908 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:40.903+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:40.945 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:40.945 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:40.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:40.970 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:40.966+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:41.071 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:41.066+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:41.090 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:41.090 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:41.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:41.172 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:41.167+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:41.177 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.174+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:41.177 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.174+0000 7f8654155640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:01:41.177 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.174+0000 7f8654155640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5,6,7 to read 0,1,4,5 2024-09-15T21:01:41.273 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:41.266+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:41.278 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.274+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6,7 to read 0,1,4,5 2024-09-15T21:01:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:41.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:41.374 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.370+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:41.378 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.374+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,4,5 2024-09-15T21:01:41.390 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.386+0000 7f8654155640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:41.390 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.386+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,4,5 2024-09-15T21:01:41.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:41.464 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:41.464 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:41.475 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:41.474+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:41.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:41.490 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.486+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,7 to read 0,1,4,5 2024-09-15T21:01:41.490 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.486+0000 7f8654155640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,4,5 2024-09-15T21:01:41.542 INFO:tasks.rados.rados.0.smithi027.stdout:367: finishing write tid 1 to smithi02732947-16 2024-09-15T21:01:41.548 INFO:tasks.rados.rados.0.smithi027.stdout:367: finishing write tid 2 to smithi02732947-16 2024-09-15T21:01:41.576 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:41.570+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:41.579 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:41.575+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,4,5 2024-09-15T21:01:41.580 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:41.580 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:41.592 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.590+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,4,5 2024-09-15T21:01:41.592 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.590+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,4,5 2024-09-15T21:01:41.592 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.590+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,7 to read 0,1,4,5 2024-09-15T21:01:41.600 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.594+0000 7f865815d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,7 to read 0,1,4,5 2024-09-15T21:01:41.607 INFO:tasks.rados.rados.0.smithi027.stdout:389: expect (ObjNum 141 snap 32 seq_num 141) 2024-09-15T21:01:41.608 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 230572 2024-09-15T21:01:41.615 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:41.615 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:41.619 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:41.619 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:41.633 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:41.651 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:41.647+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:01:41.657 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:41.651+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,4,6 to read 0,1,4,5 2024-09-15T21:01:41.658 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:41.651+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,6 to read 0,1,4,5 2024-09-15T21:01:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:41.677 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:41.671+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:41.680 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1520095 2024-09-15T21:01:41.704 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:41.704 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:41.724 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:41.724 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:41.727 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 1490821 2024-09-15T21:01:41.768 INFO:tasks.rados.rados.0.smithi027.stdout:367: finishing write tid 3 to smithi02732947-16 2024-09-15T21:01:41.768 INFO:tasks.rados.rados.0.smithi027.stdout:367: oid 16 updating version 0 to 221 2024-09-15T21:01:41.768 INFO:tasks.rados.rados.0.smithi027.stdout:367: oid 16 updating version 221 to 222 2024-09-15T21:01:41.768 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 222 (ObjNum 144 snap 33 seq_num 144) dirty exists 2024-09-15T21:01:41.769 INFO:tasks.rados.rados.0.smithi027.stdout:367: left oid 16 (ObjNum 144 snap 33 seq_num 144) 2024-09-15T21:01:41.777 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:41.770+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.817694720999995, 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.797 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:01:41.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:01:41.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.815683476000004, 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:01:41.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:01:41.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:01:41.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.814865638000001, 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.809 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:01:41.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:01:41.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:41.814 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.309604812999993, 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.308167862999994, 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:01:41.820 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:01:41.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:01:41.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.308089718000005, 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:01:41.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:01:41.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:01:41.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.205574584999994, 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:01:41.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.111139768000001, 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:01:41.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:01:41.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:01:41.839 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.104729813000006, 2024-09-15T21:01:41.840 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.841 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.842 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:01:41.844 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.845 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.00417684, 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:41.846 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:41.847 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.848 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:01:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 61.593553251000003, 2024-09-15T21:01:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.853 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:01:41.855 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.725582742, 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70767751899999998, 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.857 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068546+0000", 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.858 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069555+0000", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010088269999999999 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069566+0000", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0834e-05 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.169651+0000", 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008542099999999 2024-09-15T21:01:41.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776159+0000", 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60650789000000005 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776223+0000", 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3752999999999998e-05 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 40.628922344999999, 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71199342899999996, 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165208+0000", 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.88e-06 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166131+0000", 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092355499999999995 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166158+0000", 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7073e-05 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166172+0000", 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4249e-05 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877145+0000", 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71097216299999999 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877199+0000", 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4509000000000003e-05 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:671 3.2s0 3:48a96439:::smithi02732947-3:head [copy-from ver 128] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 30.890183367999999, 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.82993921299999995, 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 671 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.865 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903948+0000", 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1370000000000002e-06 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.866 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903963+0000", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4987000000000001e-05 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903998+0000", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4925000000000002e-05 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904016+0000", 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8093000000000001e-05 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.867 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041691+0000", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.13767560200000001 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041959+0000", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026741900000000002 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.526650+0000", 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.484691182 2024-09-15T21:01:41.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530502+0000", 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038516790000000002 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733738+0000", 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20323656000000001 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.869 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733884+0000", 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014562900000000001 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:830 3.2s0 3:447dff51:::smithi02732947-14:head [copy-from ver 48] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 20.879055736000002, 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.92164203099999997, 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.870 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 830 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:41.871 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915073+0000", 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915079+0000", 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7559999999999996e-06 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.872 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915093+0000", 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.366e-05 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915101+0000", 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6959999999999994e-06 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316410+0000", 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40130822100000002 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.873 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316742+0000", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033288600000000001 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.525877+0000", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20913427800000001 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.634058+0000", 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.108180927 2024-09-15T21:01:41.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836242+0000", 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20218430800000001 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836714+0000", 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047230599999999997 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:963 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (57) in=7b] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.1242166029999998, 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.81706183700000001, 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 963 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.876 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.877 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669943+0000", 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2109e-05 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669947+0000", 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669955+0000", 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1890000000000001e-06 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669961+0000", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0739999999999998e-06 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.670030+0000", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9876999999999995e-05 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.670038+0000", 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4229999999999999e-06 2024-09-15T21:01:41.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.775795+0000", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105757378 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.775854+0000", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8901999999999999e-05 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.776321+0000", 2024-09-15T21:01:41.880 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046652899999999998 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.883111+0000", 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.106790338 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.486880+0000", 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.603769267 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.881 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.486973+0000", 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2867000000000004e-05 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=333, last_complete=104'93, committed=1, applied=1))", 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 4.0086421220000004, 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.802818951, 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785487+0000", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6530000000000001e-06 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785498+0000", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.097e-05 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785524+0000", 2024-09-15T21:01:41.884 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5786e-05 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785537+0000", 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.306e-05 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588290+0000", 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80275311699999996 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.885 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588305+0000", 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4365000000000001e-05 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=332, last_complete=104'91, committed=1, applied=1))", 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.895859513, 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68997068900000003, 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.886 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.887 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898269+0000", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898299+0000", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9618999999999999e-05 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898312+0000", 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3899e-05 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898321+0000", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2970000000000004e-06 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588136+0000", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68981495400000004 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588239+0000", 2024-09-15T21:01:41.889 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103473 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=333, last_complete=104'93, committed=1, applied=1))", 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8957944539999998, 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69122241100000004, 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.890 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5600000000000001e-07 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898337+0000", 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.954e-06 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898346+0000", 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1589999999999999e-06 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898353+0000", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2130000000000002e-06 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.589092+0000", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.690738572 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.589556+0000", 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046405699999999998 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.12s2 104/99 ECSubWrite(tid=175, reqid=client.4641.0:938, at_version=104'87, trim_to=0'0, roll_forward_to=103'83))", 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8590657459999997, 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.75508700799999995, 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935064+0000", 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.855e-06 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935162+0000", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7993999999999997e-05 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935192+0000", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0210999999999999e-05 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935215+0000", 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2934000000000001e-05 2024-09-15T21:01:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588730+0000", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65351474300000001 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588737+0000", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.255e-06 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.690043+0000", 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101305548 2024-09-15T21:01:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.690149+0000", 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106468 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:41.898 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:41.899 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.802+0000 7f8654155640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 18 after trimming was started 2024-09-15T21:01:41.899 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:41.790+0000 7fe63b23b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:41.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:41.901 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:41.901 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:41.901 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:41.902 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:41.902 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:41.902 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:41.903 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:41.898+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:41.913 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:41.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:41.939 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:41.965 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:41.993 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:42.001 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:41.998+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:42.043 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:42.043 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:42.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.043 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.14s0 107/99 [PushOp(3:29dd4e88:::smithi02732947-38:19, version: 91'172, data_included: [786432~32768], data_size: 32768, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:29dd4e88:::smithi02732947-38:19@91'172, size: 3276800, copy_subset: [], clone_subset: {}, snapset: 20=[]:{19: [18, 16], 1e: [1e, 1d, 1b, 1a], 20: [1f]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:42.043 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.891668+0000", 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.14937170799999999, 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.149478003, 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.044 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891668+0000", 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891668+0000", 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891668+0000", 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891668+0000", 2024-09-15T21:01:42.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891768+0000", 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9748999999999997e-05 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.891829+0000", 2024-09-15T21:01:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0584999999999997e-05 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.050 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:19)])", 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.894062+0000", 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.146977469, 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.14715798999999999, 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894062+0000", 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894062+0000", 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894065+0000", 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0999999999999998e-06 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894079+0000", 2024-09-15T21:01:42.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4453e-05 2024-09-15T21:01:42.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894113+0000", 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3655999999999998e-05 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894136+0000", 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:19)])", 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.894595+0000", 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.14644452199999999, 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.14668848900000001, 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.059 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.059 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894595+0000", 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894595+0000", 2024-09-15T21:01:42.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894597+0000", 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.544e-06 2024-09-15T21:01:42.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894607+0000", 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.8160000000000005e-06 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894633+0000", 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6186e-05 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894647+0000", 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4184e-05 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:19)])", 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.894764+0000", 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.146276087, 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.146582037, 2024-09-15T21:01:42.065 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894764+0000", 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894764+0000", 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894765+0000", 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0839999999999999e-06 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894774+0000", 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.3030000000000005e-06 2024-09-15T21:01:42.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894797+0000", 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.894817+0000", 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9826999999999998e-05 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:19)])", 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.895012+0000", 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.1460275, 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.14639391600000001, 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895012+0000", 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895012+0000", 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895014+0000", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895025+0000", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.082e-05 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895055+0000", 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9739000000000001e-05 2024-09-15T21:01:42.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.895073+0000", 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7796999999999999e-05 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:19)])", 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.994964+0000", 2024-09-15T21:01:42.073 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.046076220000000001, 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.046504574999999999, 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.994964+0000", 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.994964+0000", 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.994965+0000", 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.403e-06 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.994975+0000", 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0353999999999999e-05 2024-09-15T21:01:42.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.994996+0000", 2024-09-15T21:01:42.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0176000000000001e-05 2024-09-15T21:01:42.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.995012+0000", 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6025999999999999e-05 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.14s0 107/99 [PushReplyOp(3:29dd4e88:::smithi02732947-38:20)])", 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:41.996455+0000", 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.044585369999999999, 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.045074167999999998, 2024-09-15T21:01:42.077 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996455+0000", 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996455+0000", 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996455+0000", 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996455+0000", 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996510+0000", 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5606000000000003e-05 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:41.996539+0000", 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9266000000000001e-05 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 7 2024-09-15T21:01:42.080 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:42.082 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:42.101 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:42.095+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:42.102 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:42.095+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:42.202 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:42.199+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:42.216 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:42.216 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:42.216 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:42.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:42.323 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:42.319+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:42.404 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:42.398+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.641449215999998, 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.418 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.419 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.420 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:01:42.421 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.422 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.640347849999998, 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.424 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.425 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.426 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.427 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.428 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.640307346, 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:01:42.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:01:42.430 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.431 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.432 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.433 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.619755837, 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:01:42.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:01:42.435 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.436 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.437 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:01:42.438 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.518448981000006, 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.439 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.440 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.441 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.442 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.443 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.518408590999996, 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.445 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.446 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:42.447 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.448 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.436311554, 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.449 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.450 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.451 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:01:42.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.435501814000006, 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.453 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.454 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.455 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.456 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.bs2 32/28 ECSubRead(tid=28, to_read={3:d2bf2034:::smithi02732947-31:head=0,4096,0}, subchunks={3:d2bf2034:::smithi02732947-31:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.335570110000006, 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29321832399999997, 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:42.457 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075897+0000", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075907+0000", 2024-09-15T21:01:42.458 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.798e-06 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075929+0000", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.143e-05 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.075945+0000", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6303e-05 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368622+0000", 2024-09-15T21:01:42.459 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29267662900000002 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369115+0000", 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00049385100000000001 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 77.743793490000002, 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:01:42.460 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.461 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.462 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.463 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.464 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 58.051597020999999, 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.465 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.466 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.467 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.10.18:4 3.3s0 3:cae12ce2:::smithi02732947-37:head [assert-version v141,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 20.646980566, 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32170995400000002, 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:42.468 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764489+0000", 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.593e-06 2024-09-15T21:01:42.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764500+0000", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0463e-05 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764521+0000", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.162e-05 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764537+0000", 2024-09-15T21:01:42.470 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5603e-05 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764721+0000", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018443800000000001 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764741+0000", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9346000000000001e-05 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868783+0000", 2024-09-15T21:01:42.471 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10404185000000001 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868926+0000", 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014331200000000001 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086197+0000", 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217270729 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGRecoveryDeleteReply(3.14s0 e102,99 3:2b54c2df:::smithi02732947-23:12,71'131)", 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.3405221269999998, 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31112111399999998, 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070946+0000", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1699999999999998e-07 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070949+0000", 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.873e-06 2024-09-15T21:01:42.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070962+0000", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2306999999999999e-05 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.171073+0000", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100110852 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.374553+0000", 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20348087400000001 2024-09-15T21:01:42.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.382066+0000", 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075126910000000002 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:head)])", 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.1342441929999998, 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29599977599999999, 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.477 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277224+0000", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4300000000000001e-06 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277233+0000", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7760000000000003e-06 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277255+0000", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1759999999999998e-05 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.478 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377353+0000", 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100097648 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.572386+0000", 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19503363400000001 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.573223+0000", 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083652800000000001 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.479 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0347095790000003, 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40437275900000003, 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376759+0000", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.24e-06 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376768+0000", 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9530000000000005e-06 2024-09-15T21:01:42.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376796+0000", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8626e-05 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376830+0000", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3973000000000002e-05 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781072+0000", 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40424176099999998 2024-09-15T21:01:42.482 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781130+0000", 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8205999999999998e-05 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0340589099999997, 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.407905868, 2024-09-15T21:01:42.483 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.484 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377410+0000", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.367e-06 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377422+0000", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2376e-05 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377473+0000", 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1422999999999999e-05 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.485 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377506+0000", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2672999999999997e-05 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781907+0000", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40440068499999998 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.785314+0000", 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034073440000000001 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.486 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.377785+0000", 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0336818110000001, 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29573885599999999, 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377785+0000", 2024-09-15T21:01:42.487 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377785+0000", 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377786+0000", 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1299999999999999e-07 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.488 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377794+0000", 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.807e-06 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377833+0000", 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9357000000000003e-05 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377888+0000", 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4314999999999998e-05 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.489 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.673443+0000", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.295555332 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.673524+0000", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.1231999999999997e-05 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:42.490 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 6.0335268590000002, 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40330712499999999, 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377942+0000", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377952+0000", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9860000000000006e-06 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377979+0000", 2024-09-15T21:01:42.492 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.65e-05 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.378020+0000", 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1239999999999998e-05 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781172+0000", 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.403152332 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.493 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781247+0000", 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5136999999999996e-05 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 104/59 ECSubWrite(tid=334, reqid=osd.3.0:334, at_version=104'102, trim_to=0'0, roll_forward_to=103'100))", 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.4559308089999998, 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.303416089, 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955538+0000", 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.22e-06 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955577+0000", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9696000000000001e-05 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955601+0000", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3913e-05 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955616+0000", 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.446e-05 2024-09-15T21:01:42.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.055847+0000", 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100231608 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.055867+0000", 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9389999999999999e-05 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:42.497 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.258904+0000", 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20303726999999999 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.258952+0000", 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8532000000000002e-05 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:957 3.14s0 3:2f703215:::smithi02732947-30:head [read 0~2588672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.4553746490000004, 2024-09-15T21:01:42.498 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33119290299999998, 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 957 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.499 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956094+0000", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956099+0000", 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.57e-06 2024-09-15T21:01:42.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956115+0000", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5277e-05 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.056204+0000", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10008990700000001 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.056595+0000", 2024-09-15T21:01:42.501 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039031200000000003 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.057962+0000", 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001367098 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.287285+0000", 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.22932359299999999 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.502 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.503 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:42.503 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:42.410+0000 7fe63f243640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:42.504 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:42.505 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:42.498+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:42.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:01:42.576 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:42.571+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:42.605 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:42.599+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:42.618 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:42.611+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:42.680 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.14s7 107/99 [PushOp(3:29dd4e88:::smithi02732947-38:1e, version: 89'168, data_included: [1310720~40960], data_size: 40960, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:29dd4e88:::smithi02732947-38:1e@89'168, size: 5406720, copy_subset: [], clone_subset: {}, snapset: 20=[]:{19: [18, 16], 1e: [1e, 1d, 1b, 1a], 20: [1f]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5406720, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:42.615282+0000", 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.064204833000000003, 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.064303392000000001, 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "no flag points reached", 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:42.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.615282+0000", 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.615282+0000", 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.615284+0000", 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.778e-06 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.615332+0000", 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.7991999999999997e-05 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.615373+0000", 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1219e-05 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 1 2024-09-15T21:01:42.683 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:42.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:42.707 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:42.702+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:42.807 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:42.803+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:42.831 INFO:tasks.rados.rados.0.smithi027.stdout:365: done (8 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:366: done (7 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:367: done (6 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:372: done (5 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:385: done (4 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:387: done (3 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:389: done (2 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:390: done (1 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:391: done (0 left) 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:392: copy_from oid 11 from oid 50 current snap is 34 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 16 current snap is 34 2024-09-15T21:01:42.832 INFO:tasks.rados.rados.0.smithi027.stdout:393: seq_num 156 ranges {4308992=229376} 2024-09-15T21:01:42.866 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:42.866 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:42.866 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:42.866 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:42.866 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:42.876 INFO:tasks.rados.rados.0.smithi027.stdout:393: writing smithi02732947-16 from 4308992 to 4538368 tid 1 2024-09-15T21:01:42.876 INFO:tasks.rados.rados.0.smithi027.stdout:394: rmattr oid 2 current snap is 34 2024-09-15T21:01:42.876 INFO:tasks.rados.rados.0.smithi027.stdout:394: done (2 left) 2024-09-15T21:01:42.876 INFO:tasks.rados.rados.0.smithi027.stdout:395: delete oid 35 current snap is 34 2024-09-15T21:01:42.881 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:42.908 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:42.906+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:42.944 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:42.939+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:43.009 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:43.002+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:395: done (2 left) 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:396: read oid 46 snap -1 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:396: expect (ObjNum 147 snap 33 seq_num 147) 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:397: rollback oid 18 current snap is 34 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:rollback oid 18 to 31 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:398: read oid 30 snap -1 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:398: expect (ObjNum 124 snap 23 seq_num 124) 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:append oid 32 current snap is 34 2024-09-15T21:01:43.032 INFO:tasks.rados.rados.0.smithi027.stdout:399: seq_num 158 ranges {3555328=262144} 2024-09-15T21:01:43.035 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:43.035 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,4,5 2024-09-15T21:01:43.036 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,7 to read 0,1,4,5 2024-09-15T21:01:43.036 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,7 to read 0,1,4,5 2024-09-15T21:01:43.036 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:43.037 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,4,5 2024-09-15T21:01:43.038 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:01:43.038 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.030+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,3,4,5 to read 0,1,4,5 2024-09-15T21:01:43.044 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.039+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5 to read 0,1,4,5 2024-09-15T21:01:43.046 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.039+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,5,6 to read 0,1,4,5 2024-09-15T21:01:43.046 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.039+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6 to read 0,1,4,5 2024-09-15T21:01:43.049 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.043+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:43.055 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:43.055 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:43.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.890511661999994, 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.056 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:01:43.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:01:43.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.890469296000006, 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.062 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.063 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:01:43.068 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.890078368999994, 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.069 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:01:43.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.074 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.890039091999995, 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:01:43.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:01:43.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.850807337000006, 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:01:43.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:01:43.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:01:43.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.850084984999995, 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.085 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:01:43.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:01:43.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:01:43.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.850068493999999, 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:01:43.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.095 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.356881134000005, 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:01:43.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:01:43.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.35043967, 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.101 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.102 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:01:43.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:01:43.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:43.106 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 78.350013267999998, 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.107 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:01:43.111 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 62.91070603, 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.112 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:01:43.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:01:43.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:656 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~786432] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.195039090000002, 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88797498799999997, 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.117 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 656 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858231+0000", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.097e-06 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859347+0000", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001116105 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859370+0000", 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2289999999999998e-05 2024-09-15T21:01:43.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859380+0000", 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0771000000000001e-05 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.015842+0000", 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15646217000000001 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.016121+0000", 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027814300000000001 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.017565+0000", 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001443937 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.124892+0000", 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107327906 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.121 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.745992+0000", 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62109974199999995 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746204+0000", 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021182699999999999 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:661 3.5s0 3:a8fd5180:::smithi02732947-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:43.122 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 32.079424222, 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.979178514, 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 661 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.123 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973845+0000", 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.124 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973848+0000", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3340000000000002e-06 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973861+0000", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.34e-05 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973870+0000", 2024-09-15T21:01:43.125 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535782+0000", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56191254599999996 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535923+0000", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140882 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748675+0000", 2024-09-15T21:01:43.126 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21275177100000001 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748830+0000", 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000155682 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.953022+0000", 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20419203999999999 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:764 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~507904 in=507904b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.422057578, 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89954056699999996, 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 764 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.128 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631213+0000", 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2170000000000001e-06 2024-09-15T21:01:43.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631857+0000", 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00064422299999999997 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631889+0000", 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2246999999999998e-05 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.130 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631905+0000", 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5336e-05 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320301+0000", 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68839636000000004 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320584+0000", 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028351100000000002 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.131 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.322686+0000", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021011860000000001 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.326185+0000", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003499149 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530672+0000", 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20448741300000001 2024-09-15T21:01:43.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530751+0000", 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8924999999999994e-05 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:765 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~475136 in=475136b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.419375709000001, 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0983793850000001, 2024-09-15T21:01:43.133 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 765 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.134 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633894+0000", 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2249999999999999e-06 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.135 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634374+0000", 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048002400000000001 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634405+0000", 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1189999999999998e-05 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.136 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634421+0000", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5815999999999999e-05 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324182+0000", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68976111799999995 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324311+0000", 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000129271 2024-09-15T21:01:43.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325308+0000", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099730799999999992 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.430267+0000", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10495866700000001 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732131+0000", 2024-09-15T21:01:43.138 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30186411099999999 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732272+0000", 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140655 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:766 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~98304] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.418738941000001, 2024-09-15T21:01:43.139 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0981193570000001, 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 766 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634531+0000", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4500000000000001e-06 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634708+0000", 2024-09-15T21:01:43.141 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000177104 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634721+0000", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.359e-05 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634730+0000", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9020000000000005e-06 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427504+0000", 2024-09-15T21:01:43.142 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.79277363899999997 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427699+0000", 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019504099999999999 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.429045+0000", 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001346564 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.143 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.631182+0000", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202136547 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732474+0000", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101292148 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732649+0000", 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000174372 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.144 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:767 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.418518481, 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.709944165, 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.145 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 767 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4499999999999996e-07 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634759+0000", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8650000000000007e-06 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634776+0000", 2024-09-15T21:01:43.147 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6549e-05 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634783+0000", 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5340000000000002e-06 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935746+0000", 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3009621600000001 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.148 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935934+0000", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000188035 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.936723+0000", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00078927499999999998 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.143378+0000", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20665556199999999 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.149 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344471+0000", 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20109264199999999 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344694+0000", 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022269799999999999 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:954 3.16s0 3:6f7bd2df:::smithi02732947-21:head [append 0~49152] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:43.150 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.3836493299999999, 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.411478311, 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 954 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:43.151 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669622+0000", 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.05e-06 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.152 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669745+0000", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012288199999999999 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669782+0000", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6863000000000003e-05 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669806+0000", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4646000000000001e-05 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.153 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670145+0000", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000338759 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670209+0000", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4267e-05 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.684006+0000", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013796691999999999 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:43.154 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.688884+0000", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0048775609999999999 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.841823+0000", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1529394239999999 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.842689+0000", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00086572999999999999 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.155 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.848153+0000", 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054642909999999996 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.858398+0000", 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010245216 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.079481+0000", 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.221082785 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.081097+0000", 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001616145 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:955 3.16s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (57) in=7b] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.3834041829999997, 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4147843039999999, 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.157 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 955 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.158 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669866+0000", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.764e-06 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669876+0000", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0159e-05 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.159 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669905+0000", 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8649999999999998e-05 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669923+0000", 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8580000000000002e-05 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670481+0000", 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055759699999999995 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.160 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670592+0000", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011093499999999999 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.689642+0000", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019049916 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.690193+0000", 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055138400000000003 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.161 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.851054+0000", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1608608929999999 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.852392+0000", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013381720000000001 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.855581+0000", 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031888300000000001 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.162 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.859002+0000", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034214639999999999 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.081944+0000", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.22294212999999999 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.084648+0000", 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027038299999999999 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.163 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.6.17:2 3.13s1 3:cdcabce2:::smithi02732947-35:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.2582696059999998, 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0519870250000001, 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.164 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795002+0000", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5370000000000002e-06 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.165 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795017+0000", 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4742999999999999e-05 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795047+0000", 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0695999999999999e-05 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795072+0000", 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4983000000000001e-05 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.166 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795419+0000", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00034657999999999998 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.801594+0000", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061751319999999998 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.846193+0000", 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0445987290000001 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.167 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.846986+0000", 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00079262500000000004 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.168 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:43.169 INFO:tasks.rados.rados.0.smithi027.stdout:399: writing smithi02732947-32 from 3555328 to 3817472 tid 1 2024-09-15T21:01:43.169 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 6 2024-09-15T21:01:43.170 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.130+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,4,5 2024-09-15T21:01:43.170 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.130+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,5 to read 0,1,4,5 2024-09-15T21:01:43.170 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.134+0000 7f885b28c640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,5 to read 0,1,4,5 2024-09-15T21:01:43.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.143+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.147+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.147+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.147+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.147+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.155+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.155+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.155+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6,7 to read 0,1,4,5 2024-09-15T21:01:43.172 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.155+0000 7f5af2c03640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:01:43.173 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.163+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,4,5 2024-09-15T21:01:43.173 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:43.163+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:43.255 INFO:tasks.rados.rados.0.smithi027.stdout:393: finishing write tid 1 to smithi02732947-16 2024-09-15T21:01:43.270 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.263+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,6,7 to read 0,1,4,5 2024-09-15T21:01:43.270 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:43.262+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:43.285 INFO:tasks.rados.rados.0.smithi027.stdout:392: finishing copy_from racing read to smithi02732947-11 2024-09-15T21:01:43.336 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.330+0000 7f885f294640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,3,5 to read 0,1,4,5 2024-09-15T21:01:43.337 INFO:tasks.rados.rados.0.smithi027.stdout:396: expect (ObjNum 147 snap 33 seq_num 147) 2024-09-15T21:01:43.337 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:43.340 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 41172 2024-09-15T21:01:43.341 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 29371 2024-09-15T21:01:43.342 INFO:tasks.rados.rados.0.smithi027.stdout:396: done (5 left) 2024-09-15T21:01:43.342 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 5 2024-09-15T21:01:43.343 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:43.338+0000 7fe63ca3e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:43.346 INFO:tasks.rados.rados.0.smithi027.stdout:397: finishing rollback tid 0 to smithi02732947-18 2024-09-15T21:01:43.347 INFO:tasks.rados.rados.0.smithi027.stdout:397: finishing rollback tid 1 to smithi02732947-18 2024-09-15T21:01:43.347 INFO:tasks.rados.rados.0.smithi027.stdout:397: finishing rollback tid 2 to smithi02732947-18 2024-09-15T21:01:43.347 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 18 v 106 (ObjNum 120 snap 22 seq_num 120) dirty exists 2024-09-15T21:01:43.347 INFO:tasks.rados.rados.0.smithi027.stdout:397: done (4 left) 2024-09-15T21:01:43.347 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 4 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.17s5 108/99 [PushOp(3:e8b6cabe:::smithi02732947-43:1f, version: 95'10, data_included: [0~180224], data_size: 180224, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:e8b6cabe:::smithi02732947-43:1f@95'10, size: 720896, copy_subset: [], clone_subset: {}, snapset: 22=[]:{1f: [1f, 1e, 1d, 1b, 1a, 18, 16], 22: [22, 21]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 720896, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:43.342139+0000", 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.013460880999999999, 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013545102999999999, 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:43.357 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342139+0000", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342139+0000", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342141+0000", 2024-09-15T21:01:43.358 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5399999999999998e-06 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342546+0000", 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040438200000000001 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342596+0000", 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0118999999999999e-05 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.359 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.342645+0000", 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8919999999999999e-05 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.14s5 108/99 [PushOp(3:29dd4e88:::smithi02732947-38:20, version: 93'176, data_included: [1310720~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:29dd4e88:::smithi02732947-38:20@93'176, size: 6471680, copy_subset: [], clone_subset: {}, snapset: 20=[]:{19: [18, 16], 1e: [1e, 1d, 1b, 1a], 20: [1f]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6291456, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:43.344782+0000", 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.010817571, 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.011036018, 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:43.360 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.344782+0000", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.344782+0000", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.344784+0000", 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0169999999999999e-06 2024-09-15T21:01:43.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.345008+0000", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022352400000000001 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.345071+0000", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.3048999999999999e-05 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.345114+0000", 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.3797000000000001e-05 2024-09-15T21:01:43.362 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs6 108/59 ECSubWrite(tid=344, reqid=client.4641.0:1011, at_version=0'0, trim_to=0'0, roll_forward_to=108'106))", 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:43.347280+0000", 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 0.0083198490000000007, 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.008610408, 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "queued for pg", 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.363 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347280+0000", 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347280+0000", 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347281+0000", 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.533e-06 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.364 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347301+0000", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9579999999999999e-05 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347331+0000", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.9799999999999999e-05 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.347345+0000", 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4525999999999999e-05 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.366 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.366 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2024-09-15T21:01:43.366 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 3 2024-09-15T21:01:43.366 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:43.366 INFO:tasks.rados.rados.0.smithi027.stdout:393: finishing write tid 2 to smithi02732947-16 2024-09-15T21:01:43.370 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:43.362+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:43.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:43.380 INFO:tasks.rados.rados.0.smithi027.stdout:398: expect (ObjNum 124 snap 23 seq_num 124) 2024-09-15T21:01:43.380 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:43.419 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 305268 2024-09-15T21:01:43.450 INFO:tasks.rados.rados.0.smithi027.stdout:seeking to 0 2024-09-15T21:01:43.472 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:43.466+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:43.478 INFO:tasks.rados.rados.0.smithi027.stdout:398: done (3 left) 2024-09-15T21:01:43.479 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 3 2024-09-15T21:01:43.491 INFO:tasks.rados.rados.0.smithi027.stdout:392: finishing copy_from to smithi02732947-11 2024-09-15T21:01:43.491 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 11 v 67 (ObjNum 121 snap 23 seq_num 121) dirty exists 2024-09-15T21:01:43.491 INFO:tasks.rados.rados.0.smithi027.stdout:392: done (2 left) 2024-09-15T21:01:43.491 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 2 2024-09-15T21:01:43.502 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:43.502 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:43.503 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:43.503 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:43.503 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:43.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:43.566 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.559+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,4,5 2024-09-15T21:01:43.566 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.559+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4 to read 0,1,4,5 2024-09-15T21:01:43.566 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.563+0000 7f61cfb42640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5 to read 0,1,4,5 2024-09-15T21:01:43.567 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:43.563+0000 7f61d3b4a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,4,5 to read 0,1,4,5 2024-09-15T21:01:43.573 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:43.567+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:43.582 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:43.575+0000 7f5af6c0b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:43.646 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.362503305000004, 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.647 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.648 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:01:43.649 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.650 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.360772812999997, 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.651 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.652 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:01:43.653 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.654 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.359525235000007, 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.655 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.656 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:01:43.657 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.658 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.345641130999994, 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.660 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:01:43.661 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.662 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.261759131000005, 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.663 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.664 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.665 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.666 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.159312182999997, 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.667 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.668 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:01:43.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:01:43.670 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.671 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.158393285000002, 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.672 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.673 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.674 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:01:43.675 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 79.155870015999994, 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.676 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:01:43.677 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.678 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.680 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 74.063976201000003, 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.681 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:01:43.682 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.683 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.684 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 74.061063520999994, 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.685 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.686 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.687 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:01:43.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.689 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 63.179514238000003, 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.690 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:01:43.691 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.692 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.693 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 52.774392837000001, 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.694 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.695 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:01:43.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:01:43.697 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:502 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.002947872, 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30380249300000001, 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 502 2024-09-15T21:01:43.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.700 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640008+0000", 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8420000000000001e-06 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640020+0000", 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2067e-05 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640045+0000", 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4406999999999999e-05 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.701 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640057+0000", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1917e-05 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640154+0000", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6767000000000004e-05 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640165+0000", 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.137e-05 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.702 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841240+0000", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20107475499999999 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841489+0000", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024974399999999999 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.943808+0000", 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102318624 2024-09-15T21:01:43.703 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:503 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.002874272, 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308825238, 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.704 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 503 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.705 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640080+0000", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4600000000000005e-07 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640085+0000", 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9250000000000003e-06 2024-09-15T21:01:43.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640092+0000", 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3019999999999997e-06 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640096+0000", 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1919999999999998e-06 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.707 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740388+0000", 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100292057 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740400+0000", 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1420000000000001e-05 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845387+0000", 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104987575 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.708 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845666+0000", 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000279158 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.948904+0000", 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103238063 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:504 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:43.709 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 47.00282576, 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409270668, 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 504 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.710 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:43.711 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.46e-07 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640133+0000", 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4739999999999999e-06 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640145+0000", 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2673e-05 2024-09-15T21:01:43.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640151+0000", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2829999999999998e-06 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740469+0000", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100317608 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740477+0000", 2024-09-15T21:01:43.713 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2719999999999997e-06 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.846778+0000", 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106300532 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.847035+0000", 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025722499999999998 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.714 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:57.049398+0000", 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20236315499999999 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:770 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 28.00818426, 2024-09-15T21:01:43.715 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30194747500000002, 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 770 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634770+0000", 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6300000000000004e-07 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.717 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634775+0000", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8219999999999997e-06 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634791+0000", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5994999999999998e-05 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734865+0000", 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10007408399999999 2024-09-15T21:01:43.718 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735010+0000", 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014533800000000001 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735283+0000", 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027298399999999999 2024-09-15T21:01:43.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.936717+0000", 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20143348899999999 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:960 3.as0 3:5c9d63ab:::smithi02732947-34:head [copy-from ver 0] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.9732628720000003, 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.177642012, 2024-09-15T21:01:43.720 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 960 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669694+0000", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8999999999999999e-06 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669712+0000", 2024-09-15T21:01:43.722 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7416000000000001e-05 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669752+0000", 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0363e-05 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669771+0000", 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8363999999999999e-05 2024-09-15T21:01:43.723 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669936+0000", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016557400000000001 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669959+0000", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3289999999999999e-05 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.775174+0000", 2024-09-15T21:01:43.724 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.105214328 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.775493+0000", 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031928600000000001 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847333+0000", 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.071839491 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:961 3.as0 3:5c9d63ab:::smithi02732947-34:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.9731456600000001, 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.17811054, 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 961 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.726 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669809+0000", 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2130000000000001e-06 2024-09-15T21:01:43.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669814+0000", 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1329999999999998e-06 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669829+0000", 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.543e-05 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.728 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669838+0000", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0000000000000002e-06 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670162+0000", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032351999999999999 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670173+0000", 2024-09-15T21:01:43.729 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1473e-05 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.778753+0000", 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.1085791 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.778905+0000", 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000152731 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.730 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847570+0000", 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0686646740000001 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847745+0000", 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017489799999999999 2024-09-15T21:01:43.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847918+0000", 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000173368 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 104/96 ECSubWrite(tid=180, reqid=client.4641.0:985, at_version=104'59, trim_to=0'0, roll_forward_to=103'57))", 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:43.733 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.9706426559999999, 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30899451999999999, 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.734 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672312+0000", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.026e-06 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672411+0000", 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9495999999999997e-05 2024-09-15T21:01:43.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672437+0000", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5420000000000001e-05 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672459+0000", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1970000000000001e-05 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.778474+0000", 2024-09-15T21:01:43.736 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106015363 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.778497+0000", 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.334e-05 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981218+0000", 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20272069600000001 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.737 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981305+0000", 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7208999999999998e-05 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 104/96 ECSubWrite(tid=181, reqid=client.4641.0:986, at_version=104'60, trim_to=0'0, roll_forward_to=103'57))", 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 4.9704458049999998, 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308874965, 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:43.738 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:43.739 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.13e-07 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672536+0000", 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8223999999999999e-05 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672560+0000", 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3866000000000002e-05 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.740 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672579+0000", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8739000000000001e-05 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.781009+0000", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10842980000000001 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.781032+0000", 2024-09-15T21:01:43.741 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2716000000000001e-05 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981316+0000", 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.200284781 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981383+0000", 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6225999999999999e-05 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.742 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.743 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:43.743 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:43.743 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:43.744 INFO:tasks.rados.rados.0.smithi027.stdout:393: finishing write tid 3 to smithi02732947-16 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout:393: oid 16 updating version 0 to 225 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout:393: oid 16 version 225 is already newer than 224 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 16 v 225 (ObjNum 156 snap 34 seq_num 156) dirty exists 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout:393: left oid 16 (ObjNum 156 snap 34 seq_num 156) 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout:393: done (1 left) 2024-09-15T21:01:43.745 INFO:tasks.rados.rados.0.smithi027.stdout: waiting on 1 2024-09-15T21:01:43.746 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:43.746 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:43.742+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:43.784 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is set 2024-09-15T21:01:43.855 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:43.850+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:43.927 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:43.927 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:43.928 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:43.928 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:43.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:43.946 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:43.938+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:44.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:01:44.063 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:44.058+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:44.070 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:44.071 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:44.071 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:44.071 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:44.071 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:44.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:44.149 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:44.143+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:44.180 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:44.180 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:44.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:44.197 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:44.214 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 103.77312954600001, 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.215 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:44.216 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:01:44.217 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.218 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.219 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 103.77305148000001, 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.221 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.222 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.223 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.224 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.225 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:01:44.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:01:44.227 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 99.982595774999993, 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.228 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:01:44.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:01:44.231 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:01:44.232 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 99.982425731999996, 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.234 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.235 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.236 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.237 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.238 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:437 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~540672] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.582006411000002, 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54912561999999998, 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 437 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.240 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628861+0000", 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7499999999999999e-06 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.241 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629602+0000", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142900000000005 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629633+0000", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0453000000000001e-05 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629649+0000", 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5923e-05 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.242 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629768+0000", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119453 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.630003+0000", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023483799999999999 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.632434+0000", 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024312940000000001 2024-09-15T21:01:44.243 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.636553+0000", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041183879999999997 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177838+0000", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54128565900000003 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177984+0000", 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014543299999999999 2024-09-15T21:01:44.244 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:438 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~229376] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.579437714999997, 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54706032999999998, 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.245 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 438 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.246 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631429+0000", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2630000000000002e-06 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631860+0000", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000431261 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631886+0000", 2024-09-15T21:01:44.247 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6230000000000001e-05 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631907+0000", 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633475+0000", 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015675870000000001 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.248 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633636+0000", 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016115899999999999 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.635458+0000", 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018221299999999999 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.637212+0000", 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017535420000000001 2024-09-15T21:01:44.249 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178395+0000", 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54118301599999996 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178487+0000", 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2262999999999994e-05 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.250 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:439 3.ds0 3:b220672c:::smithi02732947-15:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.578911214999998, 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.85063377100000004, 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 439 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631956+0000", 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-06 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.252 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631966+0000", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0047000000000001e-05 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631987+0000", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1253000000000001e-05 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732106+0000", 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011937999999999 2024-09-15T21:01:44.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179003+0000", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44689677 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179116+0000", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112794 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179656+0000", 2024-09-15T21:01:44.254 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054039500000000005 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180543+0000", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088661899999999995 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482437+0000", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30189397699999998 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482587+0000", 2024-09-15T21:01:44.255 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015026300000000001 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:440 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.478714293000003, 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.75401734600000003, 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.256 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 440 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732153+0000", 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.717e-06 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732167+0000", 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.382e-05 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.258 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732202+0000", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5553000000000003e-05 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732225+0000", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2894000000000001e-05 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180004+0000", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44777894899999998 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:44.259 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180092+0000", 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7705000000000001e-05 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483149+0000", 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30305710299999999 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483259+0000", 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109998 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.260 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.486168+0000", 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0029086070000000001 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:652 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.385532218000002, 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50904735300000004, 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 652 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.262 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825336+0000", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2660000000000002e-06 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825348+0000", 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2327000000000001e-05 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.263 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825385+0000", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6918999999999997e-05 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825407+0000", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2560000000000001e-05 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825508+0000", 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000100647 2024-09-15T21:01:44.264 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825615+0000", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000106795 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.826443+0000", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00082777899999999995 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:44.265 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044140749999999999 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334247+0000", 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50339011700000003 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334380+0000", 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013286799999999999 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:653 3.ds0 3:b220672c:::smithi02732947-15:head [rollback 16] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.385415928999997, 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50927399799999995, 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 653 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.267 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825450+0000", 2024-09-15T21:01:44.268 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3799999999999999e-06 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825459+0000", 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9959999999999999e-06 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825480+0000", 2024-09-15T21:01:44.269 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0930000000000001e-05 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825502+0000", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.213e-05 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827235+0000", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001732691 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827493+0000", 2024-09-15T21:01:44.270 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000258139 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.828697+0000", 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012043609999999999 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002160201 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.271 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334605+0000", 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50374753100000003 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334723+0000", 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000117639 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:654 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:44.272 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.385335099999999, 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50976337999999999, 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 654 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:44.273 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825531+0000", 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3030000000000001e-06 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.274 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825536+0000", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5670000000000002e-06 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825555+0000", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8680000000000001e-05 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825570+0000", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4715999999999999e-05 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.275 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830061+0000", 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044918199999999997 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830150+0000", 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8658000000000004e-05 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830574+0000", 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042412199999999998 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.276 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.931412+0000", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100838144 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335070+0000", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40365782900000002 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335293+0000", 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022254100000000001 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.277 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.3163245899999998, 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50728277399999999, 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.278 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894542+0000", 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.279 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895468+0000", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092661499999999999 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895495+0000", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.686e-05 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.995597+0000", 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100101948 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.280 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401647+0000", 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40604999000000003 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401823+0000", 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000175351 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.281 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=1))", 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.3104711050000004, 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70345572000000001, 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.282 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900395+0000", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.553e-06 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900696+0000", 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030154100000000001 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.283 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900721+0000", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5018999999999999e-05 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000802+0000", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100080497 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.603506+0000", 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60270436100000002 2024-09-15T21:01:44.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.603849+0000", 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00034274899999999999 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.2132071340000001, 2024-09-15T21:01:44.285 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81764706300000001, 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.286 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997658+0000", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998063+0000", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040452199999999999 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998087+0000", 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4320000000000001e-05 2024-09-15T21:01:44.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998105+0000", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7938999999999999e-05 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814944+0000", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81683928900000002 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815304+0000", 2024-09-15T21:01:44.288 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035994599999999999 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.2099237030000003, 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50637170899999995, 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.289 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000942+0000", 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2780000000000001e-06 2024-09-15T21:01:44.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001218+0000", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027612999999999997 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001230+0000", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.201e-05 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001237+0000", 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7750000000000004e-06 2024-09-15T21:01:44.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.503166+0000", 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50192909600000002 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.507312+0000", 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041464199999999996 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.124971156, 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72889100799999995, 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.293 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085895+0000", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086206+0000", 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031062599999999999 2024-09-15T21:01:44.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086231+0000", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5363000000000001e-05 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086248+0000", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.719e-05 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814488+0000", 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72824028500000004 2024-09-15T21:01:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814784+0000", 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00029581699999999998 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s4 103/96 ECSubWrite(tid=394, reqid=osd.5.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=103'223))", 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1245888730000004, 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.93721581600000003, 2024-09-15T21:01:44.296 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.297 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086277+0000", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.443e-06 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086302+0000", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4885999999999999e-05 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086321+0000", 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8870000000000001e-05 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.298 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086340+0000", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8836000000000001e-05 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815358+0000", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72901870899999999 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815372+0000", 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4073999999999999e-05 2024-09-15T21:01:44.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.023407+0000", 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.208034999 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.023491+0000", 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3999000000000007e-05 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.300 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1152727660000004, 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82567823699999998, 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.301 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095593+0000", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.133e-06 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095748+0000", 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015470100000000001 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.302 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095763+0000", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299999999999999e-05 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095770+0000", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5970000000000003e-06 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.914691+0000", 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81892052100000001 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.303 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.921270+0000", 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065789849999999999 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.1139371689999997, 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70924067599999996, 2024-09-15T21:01:44.304 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.305 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096929+0000", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8199999999999999e-06 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097251+0000", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032222400000000002 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097287+0000", 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6096000000000001e-05 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.306 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097302+0000", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5053e-05 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.805802+0000", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70849910699999996 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.806168+0000", 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036637600000000002 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.307 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:1004 3.15s2 3:ad5ec547:::smithi02732947-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e108)", 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.2766622329999999, 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.73611654900000001, 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:44.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1004 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:44.309 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934204+0000", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.404e-06 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934210+0000", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0100000000000001e-06 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934235+0000", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5602e-05 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:44.310 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934254+0000", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.836e-05 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.942267+0000", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080132669999999993 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.942525+0000", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025793399999999997 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:44.311 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.461573+0000", 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51904797800000002 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.462052+0000", 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00047958300000000001 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.670319+0000", 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20826641100000001 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.312 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.313 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:44.313 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:44.313 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:44.314 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:44.314 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:44.315 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:44.315 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:44.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:44.316 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:44.316 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:44.316 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:44.317 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:44.317 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:44.318 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:44.318 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:44.306+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:44.334 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:44.334 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:44.353 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:44.353 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:44.373 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:44.367+0000 7fde76913640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:44.374 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:44.375 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:44.396 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:44.415 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:44.411+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:44.451 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:44.451 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:44.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:44.516 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:44.514+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:44.518 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:44.511+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:44.583 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:44.583 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:44.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:44.639 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:44.638+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:44.718 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:44.711+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:44.764 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:44.764 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:44.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:44.812 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-15T21:01:44.812 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:01:44.813 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-15T21:01:44.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:01:44.823 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:44.822+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:44.899 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:44.899 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:44.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:44.920 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:44.914+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:44.976 INFO:tasks.rados.rados.0.smithi027.stdout:399: finishing write tid 1 to smithi02732947-32 2024-09-15T21:01:45.021 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:45.018+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:45.076 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.076 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:45.131 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:45.130+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:45.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:01:45.165 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-15T21:01:44.792858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":171786803,"num_objects":111,"num_object_clones":63,"num_object_copies":864,"num_objects_missing_on_primary":45,"num_objects_missing":45,"num_objects_degraded":303,"num_objects_misplaced":37,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":7,"num_read":329,"num_read_kb":130351,"num_write":644,"num_write_kb":114536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":78266406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2034,"ondisk_log_size":2034,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":287,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":911284,"kb_used_data":452336,"kb_used_omap":18,"kb_used_meta":458669,"kb_avail":1131550796,"statfs":{"total":1159641169920,"available":1158708015104,"internally_reserved":0,"allocated":463192064,"data_stored":677261914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":469677371},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,1,3,5,8,4,4,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4210688,"num_objects":6,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":-12,"num_objects_missing":-12,"num_objects_degraded":-84,"num_objects_misplaced":-30,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":-1,"num_read":5,"num_read_kb":28311,"num_write":8,"num_write_kb":-2728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":11247616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.243185"},"pg_stats":[{"pgid":"3.19","version":"105'81","reported_seq":41,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.587100+0000","last_change":"2024-09-15T21:01:34.151777+0000","last_active":"2024-09-15T21:01:40.587100+0000","last_peered":"2024-09-15T21:01:40.587100+0000","last_clean":"2024-09-15T21:01:31.482416+0000","last_became_active":"2024-09-15T21:01:34.142050+0000","last_became_peered":"2024-09-15T21:01:34.142050+0000","last_unstale":"2024-09-15T21:01:40.587100+0000","last_undegraded":"2024-09-15T21:01:33.688690+0000","last_fullsized":"2024-09-15T21:01:33.635298+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":6,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7061504,"num_objects":5,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":10,"num_objects_missing":10,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":1632,"num_write":60,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16228352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4,10,6,7,11,2],"acting":[5,3,4,2147483647,6,2147483647,2147483647,2],"avail_no_missing":["1(5)","4(6)","5(4)","6(5)","7(2)","8(0)","9(3)","10(7)","11(1)"],"object_location_counts":[{"shards":"1(5),4(6),5(4),7(2),8(0),9(3),10(7),11(1)","objects":4},{"shards":"2(7),3(1),4(2),5(0),6(4)","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"91'22","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.557639+0000","last_change":"2024-09-15T21:01:37.672921+0000","last_active":"2024-09-15T21:01:40.557639+0000","last_peered":"2024-09-15T21:01:40.557639+0000","last_clean":"2024-09-15T21:01:40.557639+0000","last_became_active":"2024-09-15T21:01:35.907012+0000","last_became_peered":"2024-09-15T21:01:35.907012+0000","last_unstale":"2024-09-15T21:01:40.557639+0000","last_undegraded":"2024-09-15T21:01:40.557639+0000","last_fullsized":"2024-09-15T21:01:40.557639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":5,"last_scrub":"91'22","last_scrub_stamp":"2024-09-15T21:01:37.549425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.549425+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:52.111036+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0.0010840400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,0,2,8,7,4,5],"acting":[3,9,0,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.17","version":"105'16","reported_seq":49,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.552253+0000","last_change":"2024-09-15T21:01:34.144972+0000","last_active":"2024-09-15T21:01:40.552253+0000","last_peered":"2024-09-15T21:01:40.552253+0000","last_clean":"2024-09-15T21:01:29.262372+0000","last_became_active":"2024-09-15T21:01:34.041177+0000","last_became_peered":"2024-09-15T21:01:34.041177+0000","last_unstale":"2024-09-15T21:01:40.552253+0000","last_undegraded":"2024-09-15T21:01:33.630775+0000","last_fullsized":"2024-09-15T21:01:33.628305+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":5,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":745472,"num_objects":4,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":7,"num_read_kb":1733,"num_write":25,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,2147483647,3,2147483647,2147483647,2,1],"avail_no_missing":["0(3)","1(7)","3(0)","4(6)","7(2)","9(5)","10(4)","11(1)"],"object_location_counts":[{"shards":"0(0),1(7),2(6),3(3),5(1)","objects":3},{"shards":"0(3),1(7),3(0),4(6),7(2),9(5),10(4),11(1)","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"1"}]},{"pgid":"3.16","version":"105'44","reported_seq":41,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.567474+0000","last_change":"2024-09-15T21:01:34.334237+0000","last_active":"2024-09-15T21:01:40.567474+0000","last_peered":"2024-09-15T21:01:40.567474+0000","last_clean":"2024-09-15T21:01:31.471353+0000","last_became_active":"2024-09-15T21:01:34.224102+0000","last_became_peered":"2024-09-15T21:01:34.224102+0000","last_unstale":"2024-09-15T21:01:40.567474+0000","last_undegraded":"2024-09-15T21:01:33.684006+0000","last_fullsized":"2024-09-15T21:01:33.636792+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002370137,"stat_sum":{"num_bytes":1712128,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":13,"num_read_kb":1915,"num_write":30,"num_write_kb":1832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2859008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,3,8,2,10,6],"acting":[5,2147483647,4,3,2147483647,2,2147483647,6],"avail_no_missing":["0(0)","4(2)","5(7)","7(3)","8(1)","9(6)","10(5)","11(4)"],"object_location_counts":[{"shards":"0(0),4(2),5(7),7(3),8(1),9(6),10(5),11(4)","objects":3},{"shards":"2(5),3(3),4(2),5(0),6(7)","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.15","version":"107'222","reported_seq":62,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded","last_fresh":"2024-09-15T21:01:41.548124+0000","last_change":"2024-09-15T21:01:34.224389+0000","last_active":"2024-09-15T21:01:41.548124+0000","last_peered":"2024-09-15T21:01:41.548124+0000","last_clean":"2024-09-15T21:01:28.092733+0000","last_became_active":"2024-09-15T21:01:34.209161+0000","last_became_peered":"2024-09-15T21:01:34.209161+0000","last_unstale":"2024-09-15T21:01:41.548124+0000","last_undegraded":"2024-09-15T21:01:33.755887+0000","last_fullsized":"2024-09-15T21:01:33.742021+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21315584,"num_objects":11,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":13,"num_objects_missing":13,"num_objects_degraded":104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":36,"num_read_kb":3253,"num_write":90,"num_write_kb":24112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":15597568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[2147483647,2147483647,7,4,2,6,0,2147483647],"avail_no_missing":["2(2)","3(1)","5(0)","6(3)","7(4)","8(5)","10(7)","11(6)"],"object_location_counts":[{"shards":"0(6),2(4),4(3),6(5),7(2)","objects":5},{"shards":"2(2),3(1),5(0),6(3),7(4),8(5),10(7),11(6)","objects":6}],"blocked_by":[],"up_primary":8,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.14","version":"102'181","reported_seq":133,"reported_epoch":107,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:01:40.661513+0000","last_change":"2024-09-15T21:01:37.760200+0000","last_active":"2024-09-15T21:01:40.661513+0000","last_peered":"2024-09-15T21:01:40.661513+0000","last_clean":"2024-09-15T21:01:26.982743+0000","last_became_active":"2024-09-15T21:01:33.916378+0000","last_became_peered":"2024-09-15T21:01:33.916378+0000","last_unstale":"2024-09-15T21:01:40.661513+0000","last_undegraded":"2024-09-15T21:01:40.661513+0000","last_fullsized":"2024-09-15T21:01:33.630920+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":8,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":21,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":28,"num_read_kb":19584,"num_write":47,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7127040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,2147483647,7,2147483647,2,5],"avail_no_missing":["0(7)","2(5)","5(3)","6(6)","7(4)","9(2)","10(0)","11(1)"],"object_location_counts":[{"shards":"0(7),2(5),5(3),6(6),7(4),9(2),10(0),11(1)","objects":3},{"shards":"2(6),3(2),4(0),5(7),7(4),8(1)","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.564452+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:01:40.564452+0000","last_peered":"2024-09-15T21:01:40.564452+0000","last_clean":"2024-09-15T21:01:40.564452+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:01:40.564452+0000","last_undegraded":"2024-09-15T21:01:40.564452+0000","last_fullsized":"2024-09-15T21:01:40.564452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:08.004258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'44","reported_seq":455,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.552230+0000","last_change":"2024-09-15T21:01:37.666878+0000","last_active":"2024-09-15T21:01:40.552230+0000","last_peered":"2024-09-15T21:01:40.552230+0000","last_clean":"2024-09-15T21:01:40.552230+0000","last_became_active":"2024-09-15T21:01:30.958712+0000","last_became_peered":"2024-09-15T21:01:30.958712+0000","last_unstale":"2024-09-15T21:01:40.552230+0000","last_undegraded":"2024-09-15T21:01:40.552230+0000","last_fullsized":"2024-09-15T21:01:40.552230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'41","last_scrub_stamp":"2024-09-15T21:01:37.339216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.339216+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:40.058424+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0.004198416,"stat_sum":{"num_bytes":327680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":127,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.556030+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:01:40.556030+0000","last_peered":"2024-09-15T21:01:40.556030+0000","last_clean":"2024-09-15T21:01:40.556030+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:01:40.556030+0000","last_undegraded":"2024-09-15T21:01:40.556030+0000","last_fullsized":"2024-09-15T21:01:40.556030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:32.658647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"95'11","reported_seq":426,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.558024+0000","last_change":"2024-09-15T21:01:37.673352+0000","last_active":"2024-09-15T21:01:40.558024+0000","last_peered":"2024-09-15T21:01:40.558024+0000","last_clean":"2024-09-15T21:01:40.558024+0000","last_became_active":"2024-09-15T21:01:31.161568+0000","last_became_peered":"2024-09-15T21:01:31.161568+0000","last_unstale":"2024-09-15T21:01:40.558024+0000","last_undegraded":"2024-09-15T21:01:40.558024+0000","last_fullsized":"2024-09-15T21:01:40.558024+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'11","last_scrub_stamp":"2024-09-15T21:01:35.352007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.352007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:47.669895+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0.00090138299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":193,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.589102+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:01:40.589102+0000","last_peered":"2024-09-15T21:01:40.589102+0000","last_clean":"2024-09-15T21:01:40.589102+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:01:40.589102+0000","last_undegraded":"2024-09-15T21:01:40.589102+0000","last_fullsized":"2024-09-15T21:01:40.589102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'186","reported_seq":572,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.785357+0000","last_change":"2024-09-15T21:01:38.390932+0000","last_active":"2024-09-15T21:01:42.785357+0000","last_peered":"2024-09-15T21:01:42.785357+0000","last_clean":"2024-09-15T21:01:42.785357+0000","last_became_active":"2024-09-15T21:01:31.064590+0000","last_became_peered":"2024-09-15T21:01:31.064590+0000","last_unstale":"2024-09-15T21:01:42.785357+0000","last_undegraded":"2024-09-15T21:01:42.785357+0000","last_fullsized":"2024-09-15T21:01:42.785357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'179","last_scrub_stamp":"2024-09-15T21:01:35.859573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.859573+0000","objects_scrubbed":6,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.557103+0000","scrub_duration":355,"objects_trimmed":0,"snaptrim_duration":0.70725242700000002,"stat_sum":{"num_bytes":15695872,"num_objects":7,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[10,11,9,5,7,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.551737+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:01:40.551737+0000","last_peered":"2024-09-15T21:01:40.551737+0000","last_clean":"2024-09-15T21:01:40.551737+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:01:40.551737+0000","last_undegraded":"2024-09-15T21:01:40.551737+0000","last_fullsized":"2024-09-15T21:01:40.551737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:10.241744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"106'236","reported_seq":679,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567882+0000","last_change":"2024-09-15T21:01:38.431997+0000","last_active":"2024-09-15T21:01:40.567882+0000","last_peered":"2024-09-15T21:01:40.567882+0000","last_clean":"2024-09-15T21:01:40.567882+0000","last_became_active":"2024-09-15T21:01:30.973822+0000","last_became_peered":"2024-09-15T21:01:30.973822+0000","last_unstale":"2024-09-15T21:01:40.567882+0000","last_undegraded":"2024-09-15T21:01:40.567882+0000","last_fullsized":"2024-09-15T21:01:40.567882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'219","last_scrub_stamp":"2024-09-15T21:01:35.157208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.157208+0000","objects_scrubbed":9,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:55.619851+0000","scrub_duration":211,"objects_trimmed":0,"snaptrim_duration":0.65588598099999995,"stat_sum":{"num_bytes":18317312,"num_objects":12,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.3","version":"91'46","reported_seq":426,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.562694+0000","last_change":"2024-09-15T21:01:37.767656+0000","last_active":"2024-09-15T21:01:40.562694+0000","last_peered":"2024-09-15T21:01:40.562694+0000","last_clean":"2024-09-15T21:01:40.562694+0000","last_became_active":"2024-09-15T21:01:31.264196+0000","last_became_peered":"2024-09-15T21:01:31.264196+0000","last_unstale":"2024-09-15T21:01:40.562694+0000","last_undegraded":"2024-09-15T21:01:40.562694+0000","last_fullsized":"2024-09-15T21:01:40.562694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'46","last_scrub_stamp":"2024-09-15T21:01:35.068798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.068798+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:03.665957+0000","scrub_duration":371,"objects_trimmed":0,"snaptrim_duration":0.0017916,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":212,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567156+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:01:40.567156+0000","last_peered":"2024-09-15T21:01:40.567156+0000","last_clean":"2024-09-15T21:01:40.567156+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:01:40.567156+0000","last_undegraded":"2024-09-15T21:01:40.567156+0000","last_fullsized":"2024-09-15T21:01:40.567156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"104'73","reported_seq":499,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.885290+0000","last_change":"2024-09-15T21:01:38.790030+0000","last_active":"2024-09-15T21:01:42.885290+0000","last_peered":"2024-09-15T21:01:42.885290+0000","last_clean":"2024-09-15T21:01:42.885290+0000","last_became_active":"2024-09-15T21:01:31.363252+0000","last_became_peered":"2024-09-15T21:01:31.363252+0000","last_unstale":"2024-09-15T21:01:42.885290+0000","last_undegraded":"2024-09-15T21:01:42.885290+0000","last_fullsized":"2024-09-15T21:01:42.885290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'69","last_scrub_stamp":"2024-09-15T21:01:38.158232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:38.158232+0000","objects_scrubbed":8,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:04.945407+0000","scrub_duration":1771,"objects_trimmed":0,"snaptrim_duration":0.13175252800000001,"stat_sum":{"num_bytes":15351808,"num_objects":8,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.786750+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:01:40.786750+0000","last_peered":"2024-09-15T21:01:40.786750+0000","last_clean":"2024-09-15T21:01:40.786750+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:01:40.786750+0000","last_undegraded":"2024-09-15T21:01:40.786750+0000","last_fullsized":"2024-09-15T21:01:40.786750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"104'60","reported_seq":501,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.651928+0000","last_change":"2024-09-15T21:01:37.763625+0000","last_active":"2024-09-15T21:01:40.651928+0000","last_peered":"2024-09-15T21:01:40.651928+0000","last_clean":"2024-09-15T21:01:40.651928+0000","last_became_active":"2024-09-15T21:01:31.056403+0000","last_became_peered":"2024-09-15T21:01:31.056403+0000","last_unstale":"2024-09-15T21:01:40.651928+0000","last_undegraded":"2024-09-15T21:01:40.651928+0000","last_fullsized":"2024-09-15T21:01:40.651928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'55","last_scrub_stamp":"2024-09-15T21:01:36.098483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.098483+0000","objects_scrubbed":2,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:38.253595+0000","scrub_duration":459,"objects_trimmed":0,"snaptrim_duration":0.001132278,"stat_sum":{"num_bytes":3899392,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.693150+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:01:40.693150+0000","last_peered":"2024-09-15T21:01:40.693150+0000","last_clean":"2024-09-15T21:01:40.693150+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:01:40.693150+0000","last_undegraded":"2024-09-15T21:01:40.693150+0000","last_fullsized":"2024-09-15T21:01:40.693150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"104'93","reported_seq":579,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.658064+0000","last_change":"2024-09-15T21:01:39.194412+0000","last_active":"2024-09-15T21:01:40.658064+0000","last_peered":"2024-09-15T21:01:40.658064+0000","last_clean":"2024-09-15T21:01:40.658064+0000","last_became_active":"2024-09-15T21:01:31.166834+0000","last_became_peered":"2024-09-15T21:01:31.166834+0000","last_unstale":"2024-09-15T21:01:40.658064+0000","last_undegraded":"2024-09-15T21:01:40.658064+0000","last_fullsized":"2024-09-15T21:01:40.658064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"98'85","last_scrub_stamp":"2024-09-15T21:01:36.874673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.874673+0000","objects_scrubbed":7,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:00.638345+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0.60580713200000003,"stat_sum":{"num_bytes":9224192,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":187,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567360+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:01:40.567360+0000","last_peered":"2024-09-15T21:01:40.567360+0000","last_clean":"2024-09-15T21:01:40.567360+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:01:40.567360+0000","last_undegraded":"2024-09-15T21:01:40.567360+0000","last_fullsized":"2024-09-15T21:01:40.567360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":307,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.693773+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:01:40.693773+0000","last_peered":"2024-09-15T21:01:40.693773+0000","last_clean":"2024-09-15T21:01:40.693773+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:01:40.693773+0000","last_undegraded":"2024-09-15T21:01:40.693773+0000","last_fullsized":"2024-09-15T21:01:40.693773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"104'28","reported_seq":424,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.784773+0000","last_change":"2024-09-15T21:01:38.662631+0000","last_active":"2024-09-15T21:01:42.784773+0000","last_peered":"2024-09-15T21:01:42.784773+0000","last_clean":"2024-09-15T21:01:42.784773+0000","last_became_active":"2024-09-15T21:01:31.267688+0000","last_became_peered":"2024-09-15T21:01:31.267688+0000","last_unstale":"2024-09-15T21:01:42.784773+0000","last_undegraded":"2024-09-15T21:01:42.784773+0000","last_fullsized":"2024-09-15T21:01:42.784773+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'24","last_scrub_stamp":"2024-09-15T21:01:36.086556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.086556+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:51.493292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0.096872859000000006,"stat_sum":{"num_bytes":5537792,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":12437,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.9","version":"103'79","reported_seq":529,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.555268+0000","last_change":"2024-09-15T21:01:37.894900+0000","last_active":"2024-09-15T21:01:40.555268+0000","last_peered":"2024-09-15T21:01:40.555268+0000","last_clean":"2024-09-15T21:01:40.555268+0000","last_became_active":"2024-09-15T21:01:30.773553+0000","last_became_peered":"2024-09-15T21:01:30.773553+0000","last_unstale":"2024-09-15T21:01:40.555268+0000","last_undegraded":"2024-09-15T21:01:40.555268+0000","last_fullsized":"2024-09-15T21:01:40.555268+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'77","last_scrub_stamp":"2024-09-15T21:01:35.180901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.180901+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.710717+0000","scrub_duration":503,"objects_trimmed":0,"snaptrim_duration":0.101288825,"stat_sum":{"num_bytes":4489216,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.a","version":"103'17","reported_seq":389,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.562313+0000","last_change":"2024-09-15T21:01:37.778516+0000","last_active":"2024-09-15T21:01:40.562313+0000","last_peered":"2024-09-15T21:01:40.562313+0000","last_clean":"2024-09-15T21:01:40.562313+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:01:40.562313+0000","last_undegraded":"2024-09-15T21:01:40.562313+0000","last_fullsized":"2024-09-15T21:01:40.562313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:46.196972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040306170000000002,"stat_sum":{"num_bytes":4423680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":12576,"num_write":12,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.b","version":"104'102","reported_seq":501,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.558307+0000","last_change":"2024-09-15T21:01:38.467385+0000","last_active":"2024-09-15T21:01:40.558307+0000","last_peered":"2024-09-15T21:01:40.558307+0000","last_clean":"2024-09-15T21:01:40.558307+0000","last_became_active":"2024-09-15T21:00:51.891415+0000","last_became_peered":"2024-09-15T21:00:51.891415+0000","last_unstale":"2024-09-15T21:01:40.558307+0000","last_undegraded":"2024-09-15T21:01:40.558307+0000","last_fullsized":"2024-09-15T21:01:40.558307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69506695900000004,"stat_sum":{"num_bytes":12288000,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":29767,"num_write":48,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.c","version":"104'98","reported_seq":446,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.667323+0000","last_change":"2024-09-15T21:01:37.988256+0000","last_active":"2024-09-15T21:01:40.667323+0000","last_peered":"2024-09-15T21:01:40.667323+0000","last_clean":"2024-09-15T21:01:40.667323+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:01:40.667323+0000","last_undegraded":"2024-09-15T21:01:40.667323+0000","last_fullsized":"2024-09-15T21:01:40.667323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31138399500000002,"stat_sum":{"num_bytes":10354688,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":11641,"num_write":55,"num_write_kb":10656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.d","version":"104'31","reported_seq":390,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.692768+0000","last_change":"2024-09-15T21:01:38.883235+0000","last_active":"2024-09-15T21:01:40.692768+0000","last_peered":"2024-09-15T21:01:40.692768+0000","last_clean":"2024-09-15T21:01:40.692768+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:01:40.692768+0000","last_undegraded":"2024-09-15T21:01:40.692768+0000","last_fullsized":"2024-09-15T21:01:40.692768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.118787121,"stat_sum":{"num_bytes":1687552,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":16,"num_read_kb":4922,"num_write":30,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.e","version":"104'22","reported_seq":422,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.786982+0000","last_change":"2024-09-15T21:01:37.696525+0000","last_active":"2024-09-15T21:01:42.786982+0000","last_peered":"2024-09-15T21:01:42.786982+0000","last_clean":"2024-09-15T21:01:42.786982+0000","last_became_active":"2024-09-15T21:01:25.767274+0000","last_became_peered":"2024-09-15T21:01:25.767274+0000","last_unstale":"2024-09-15T21:01:42.786982+0000","last_undegraded":"2024-09-15T21:01:42.786982+0000","last_fullsized":"2024-09-15T21:01:42.786982+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013156518000000001,"stat_sum":{"num_bytes":884736,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1258,"num_write":10,"num_write_kb":1952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2375680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":348,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.694486+0000","last_change":"2024-09-15T21:01:37.790590+0000","last_active":"2024-09-15T21:01:40.694486+0000","last_peered":"2024-09-15T21:01:40.694486+0000","last_clean":"2024-09-15T21:01:40.694486+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:01:40.694486+0000","last_undegraded":"2024-09-15T21:01:40.694486+0000","last_fullsized":"2024-09-15T21:01:40.694486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012251160000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.10","version":"93'56","reported_seq":465,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.561209+0000","last_change":"2024-09-15T21:01:37.776011+0000","last_active":"2024-09-15T21:01:40.561209+0000","last_peered":"2024-09-15T21:01:40.561209+0000","last_clean":"2024-09-15T21:01:40.561209+0000","last_became_active":"2024-09-15T21:01:35.994337+0000","last_became_peered":"2024-09-15T21:01:35.994337+0000","last_unstale":"2024-09-15T21:01:40.561209+0000","last_undegraded":"2024-09-15T21:01:40.561209+0000","last_fullsized":"2024-09-15T21:01:40.561209+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":5,"last_scrub":"93'56","last_scrub_stamp":"2024-09-15T21:01:37.452341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.452341+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:41.786020+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0.002419652,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,10,7],"acting":[6,9,5,3,0,2,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.11","version":"107'57","reported_seq":460,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.787199+0000","last_change":"2024-09-15T21:01:42.467713+0000","last_active":"2024-09-15T21:01:42.787199+0000","last_peered":"2024-09-15T21:01:42.787199+0000","last_clean":"2024-09-15T21:01:42.787199+0000","last_became_active":"2024-09-15T21:01:40.966359+0000","last_became_peered":"2024-09-15T21:01:40.966359+0000","last_unstale":"2024-09-15T21:01:42.787199+0000","last_undegraded":"2024-09-15T21:01:42.787199+0000","last_fullsized":"2024-09-15T21:01:42.787199+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-15T21:01:51.490569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.398133026,"stat_sum":{"num_bytes":4120576,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":18894,"num_write":24,"num_write_kb":4088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6905856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,8,11],"acting":[10,4,6,0,3,9,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.12","version":"104'88","reported_seq":59,"reported_epoch":107,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:01:40.651707+0000","last_change":"2024-09-15T21:01:40.332812+0000","last_active":"2024-09-15T21:01:40.651707+0000","last_peered":"2024-09-15T21:01:40.651707+0000","last_clean":"2024-09-15T21:01:25.370146+0000","last_became_active":"2024-09-15T21:01:34.052109+0000","last_became_peered":"2024-09-15T21:01:34.052109+0000","last_unstale":"2024-09-15T21:01:40.651707+0000","last_undegraded":"2024-09-15T21:01:40.651707+0000","last_fullsized":"2024-09-15T21:01:33.630656+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10862592,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":16,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":22,"num_read_kb":9511,"num_write":37,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":16113664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,8,2,5,6,11],"acting":[0,7,3,2147483647,2,5,6,2147483647],"avail_no_missing":["1(5)","3(0)","4(3)","5(4)","6(2)","8(6)","9(1)","10(7)"],"object_location_counts":[{"shards":"0(0),2(4),3(2),5(5),6(6),7(1)","objects":5},{"shards":"1(5),3(0),4(3),5(4),6(2),8(6),9(1),10(7)","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.13","version":"91'46","reported_seq":37,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded","last_fresh":"2024-09-15T21:01:40.567157+0000","last_change":"2024-09-15T21:01:34.052850+0000","last_active":"2024-09-15T21:01:40.567157+0000","last_peered":"2024-09-15T21:01:40.567157+0000","last_clean":"2024-09-15T21:01:24.965153+0000","last_became_active":"2024-09-15T21:01:34.045247+0000","last_became_peered":"2024-09-15T21:01:34.045247+0000","last_unstale":"2024-09-15T21:01:40.567157+0000","last_undegraded":"2024-09-15T21:01:33.682859+0000","last_fullsized":"2024-09-15T21:01:33.634429+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":10,"num_objects_missing":10,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8,"num_read_kb":1159,"num_write":19,"num_write_kb":6504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[2147483647,5,4,2,6,1,9,2147483647],"avail_no_missing":["0(2)","3(1)","4(0)","5(5)","6(3)","9(6)","10(7)","11(4)"],"object_location_counts":[{"shards":"0(2),3(1),4(0),5(5),6(3),9(6),10(7),11(4)","objects":6},{"shards":"1(5),2(3),4(2),5(1),6(4),9(6)","objects":-2}],"blocked_by":[],"up_primary":7,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":171196416,"num_objects":107,"num_object_clones":63,"num_object_copies":856,"num_objects_missing_on_primary":45,"num_objects_missing":45,"num_objects_degraded":303,"num_objects_misplaced":37,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":7,"num_read":253,"num_read_kb":130287,"num_write":519,"num_write_kb":112384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":78266368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454422528,"data_stored":683126784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1939,"ondisk_log_size":1939,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65876,"kb_used_data":34692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94305964,"statfs":{"total":96636764160,"available":96569307136,"internally_reserved":0,"allocated":35524608,"data_stored":53076607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411353,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105788,"kb_used_data":52396,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94266052,"statfs":{"total":96636764160,"available":96528437248,"internally_reserved":0,"allocated":53653504,"data_stored":78918815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411352,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93568,"kb_used_data":40172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94278272,"statfs":{"total":96636764160,"available":96540950528,"internally_reserved":0,"allocated":41136128,"data_stored":62173823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411351,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68340,"kb_used_data":37156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303500,"statfs":{"total":96636764160,"available":96566784000,"internally_reserved":0,"allocated":38047744,"data_stored":55534207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411351,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69860,"kb_used_data":38676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301980,"statfs":{"total":96636764160,"available":96565227520,"internally_reserved":0,"allocated":39604224,"data_stored":60691615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444056,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68340,"kb_used_data":37144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303500,"statfs":{"total":96636764160,"available":96566784000,"internally_reserved":0,"allocated":38035456,"data_stored":54039167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98804,"kb_used_data":49560,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94273036,"statfs":{"total":96636764160,"available":96535588864,"internally_reserved":0,"allocated":50749440,"data_stored":74908287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444056,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101688,"kb_used_data":48292,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94270152,"statfs":{"total":96636764160,"available":96532635648,"internally_reserved":0,"allocated":49451008,"data_stored":74445458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444055,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":28784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":29474816,"data_stored":40120959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,0,2,5,8,2,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444057,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68320,"kb_used_data":37136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303520,"statfs":{"total":96636764160,"available":96566804480,"internally_reserved":0,"allocated":38027264,"data_stored":56660607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444055,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47680,"kb_used_data":16484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324160,"statfs":{"total":96636764160,"available":96587939840,"internally_reserved":0,"allocated":16879616,"data_stored":20947602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444056,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63040,"kb_used_data":31844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308800,"statfs":{"total":96636764160,"available":96572211200,"internally_reserved":0,"allocated":32608256,"data_stored":45744767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,3,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31858688,"data_stored":46567424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15572992,"data_stored":20144128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34631680,"data_stored":52682752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36724736,"data_stored":55857152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29749248,"data_stored":41967616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":48439296,"data_stored":73936896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49979392,"data_stored":74645504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":38576128,"data_stored":60166144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":38404096,"data_stored":57462784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36761600,"data_stored":54337536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40038400,"data_stored":61575168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":53686272,"data_stored":83783680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:01:45.167 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-15T21:01:45.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 36 2024-09-15T21:01:45.251 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:45.246+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:45.258 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.258 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:45.274 INFO:tasks.rados.rados.0.smithi027.stdout:399: finishing write tid 2 to smithi02732947-32 2024-09-15T21:01:45.324 INFO:tasks.ceph.osd.8.smithi092.stderr:2024-09-15T21:01:45.322+0000 7f5a92bd8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26610) UID: 0 2024-09-15T21:01:45.395 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.395 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:45.426 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:45.418+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:45.475 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:45.471+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2 to read 0,1,4,5 2024-09-15T21:01:45.527 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:45.523+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:45.574 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.574 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:45.627 INFO:tasks.ceph.osd.4.smithi066.stderr:2024-09-15T21:01:45.623+0000 7f5b0f61c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26392) UID: 0 2024-09-15T21:01:45.712 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.712 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:45.729 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:45.726+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:45.783 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:45.779+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,4,5 2024-09-15T21:01:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 3 pg_num to 36 2024-09-15T21:01:45.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:01:45.867 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:45.862+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:45.885 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:45.879+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,4,5 2024-09-15T21:01:45.898 INFO:tasks.rados.rados.0.smithi027.stdout:399: finishing write tid 3 to smithi02732947-32 2024-09-15T21:01:45.898 INFO:tasks.rados.rados.0.smithi027.stdout:399: oid 32 updating version 0 to 144 2024-09-15T21:01:45.898 INFO:tasks.rados.rados.0.smithi027.stdout:399: oid 32 version 144 is already newer than 143 2024-09-15T21:01:45.898 INFO:tasks.rados.rados.0.smithi027.stdout:update_object_version oid 32 v 144 (ObjNum 158 snap 34 seq_num 158) dirty exists 2024-09-15T21:01:45.899 INFO:tasks.rados.rados.0.smithi027.stdout:399: left oid 32 (ObjNum 158 snap 34 seq_num 158) 2024-09-15T21:01:45.899 INFO:tasks.rados.rados.0.smithi027.stdout:399: done (0 left) 2024-09-15T21:01:45.902 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:45.902 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:45.905 INFO:tasks.rados.rados.0.smithi027.stderr:0 errors. 2024-09-15T21:01:45.905 INFO:tasks.rados.rados.0.smithi027.stderr: 2024-09-15T21:01:45.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:01:45.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:45.955 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:45.950+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:46.004 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:45.995+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:46.032 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:46.026+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:46.035 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:46.036 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:46.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:46.133 INFO:tasks.ceph.osd.11.smithi092.stderr:2024-09-15T21:01:46.129+0000 7f84d9f28640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26616) UID: 0 2024-09-15T21:01:46.173 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:46.173 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:46.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:01:46.235 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:46.230+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:46.263 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:46.263 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":111,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:01:45.791668+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T21:01:16.623312+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-15T21:00:17.994699+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":8,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":36,"pg_num_target":36,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"96","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"lrcprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":true,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":0.3756866455078125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:03.908437+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:00:54.445472+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.815936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.317862+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:00:55.983180+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:06.653717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:07.037212+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:24.672718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:01.773094+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:00.521472+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:00:55.547591+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:15.906985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.13","osds":[2147483647,5,4,2,6,1,9,2147483647]},{"pgid":"3.14","osds":[4,8,3,11,2147483647,6,2,5]},{"pgid":"3.15","osds":[2147483647,2147483647,7,4,2,6,0,2147483647]},{"pgid":"3.16","osds":[5,2147483647,4,3,2147483647,2,2147483647,6]},{"pgid":"3.19","osds":[5,3,4,2147483647,6,2147483647,2147483647,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":3},{"begin":32,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:01:46.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:01:46.311 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:46.311 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:46.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:01:46.375 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:46.370+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:46.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:01:46.465 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:46.458+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:46.496 INFO:teuthology.orchestra.run.smithi027.stdout:ERROR: (22) Invalid argument 2024-09-15T21:01:46.496 INFO:teuthology.orchestra.run.smithi027.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:01:46.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:01:46.559 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-09-15T21:01:46.554+0000 7f8876ca3640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26252) UID: 0 2024-09-15T21:01:46.577 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0: {} 2024-09-15T21:01:46.578 INFO:teuthology.orchestra.run.smithi027.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:01:46.595 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1: {} 2024-09-15T21:01:46.595 INFO:teuthology.orchestra.run.smithi027.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:01:46.612 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2: {} 2024-09-15T21:01:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:01:46.631 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3: {} 2024-09-15T21:01:46.631 INFO:teuthology.orchestra.run.smithi027.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:01:46.637 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-09-15T21:01:46.630+0000 7fe656c52640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26246) UID: 0 2024-09-15T21:01:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:01:46.652 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:01:46.652 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4: {} 2024-09-15T21:01:46.653 INFO:teuthology.orchestra.run.smithi027.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:01:46.671 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-15T21:01:45.803695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":171786803,"num_objects":111,"num_object_clones":63,"num_object_copies":864,"num_objects_missing_on_primary":45,"num_objects_missing":45,"num_objects_degraded":303,"num_objects_misplaced":37,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":7,"num_read":329,"num_read_kb":130351,"num_write":644,"num_write_kb":114536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":78266406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2034,"ondisk_log_size":2034,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":287,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":911284,"kb_used_data":452336,"kb_used_omap":18,"kb_used_meta":458669,"kb_avail":1131550796,"statfs":{"total":1159641169920,"available":1158708015104,"internally_reserved":0,"allocated":463192064,"data_stored":677261914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":469677371},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,1,3,5,8,4,4,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4210688,"num_objects":6,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":-12,"num_objects_missing":-12,"num_objects_degraded":-84,"num_objects_misplaced":-30,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":-1,"num_read":5,"num_read_kb":28311,"num_write":8,"num_write_kb":-2728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":11247616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398501"},"pg_stats":[{"pgid":"3.19","version":"105'81","reported_seq":41,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.587100+0000","last_change":"2024-09-15T21:01:34.151777+0000","last_active":"2024-09-15T21:01:40.587100+0000","last_peered":"2024-09-15T21:01:40.587100+0000","last_clean":"2024-09-15T21:01:31.482416+0000","last_became_active":"2024-09-15T21:01:34.142050+0000","last_became_peered":"2024-09-15T21:01:34.142050+0000","last_unstale":"2024-09-15T21:01:40.587100+0000","last_undegraded":"2024-09-15T21:01:33.688690+0000","last_fullsized":"2024-09-15T21:01:33.635298+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":6,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7061504,"num_objects":5,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":10,"num_objects_missing":10,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":1632,"num_write":60,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":16228352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4,10,6,7,11,2],"acting":[5,3,4,2147483647,6,2147483647,2147483647,2],"avail_no_missing":["1(5)","4(6)","5(4)","6(5)","7(2)","8(0)","9(3)","10(7)","11(1)"],"object_location_counts":[{"shards":"1(5),4(6),5(4),7(2),8(0),9(3),10(7),11(1)","objects":4},{"shards":"2(7),3(1),4(2),5(0),6(4)","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"91'22","reported_seq":68,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.557639+0000","last_change":"2024-09-15T21:01:37.672921+0000","last_active":"2024-09-15T21:01:40.557639+0000","last_peered":"2024-09-15T21:01:40.557639+0000","last_clean":"2024-09-15T21:01:40.557639+0000","last_became_active":"2024-09-15T21:01:35.907012+0000","last_became_peered":"2024-09-15T21:01:35.907012+0000","last_unstale":"2024-09-15T21:01:40.557639+0000","last_undegraded":"2024-09-15T21:01:40.557639+0000","last_fullsized":"2024-09-15T21:01:40.557639+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":5,"last_scrub":"91'22","last_scrub_stamp":"2024-09-15T21:01:37.549425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.549425+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:52.111036+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0.0010840400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,0,2,8,7,4,5],"acting":[3,9,0,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.17","version":"105'16","reported_seq":49,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.552253+0000","last_change":"2024-09-15T21:01:34.144972+0000","last_active":"2024-09-15T21:01:40.552253+0000","last_peered":"2024-09-15T21:01:40.552253+0000","last_clean":"2024-09-15T21:01:29.262372+0000","last_became_active":"2024-09-15T21:01:34.041177+0000","last_became_peered":"2024-09-15T21:01:34.041177+0000","last_unstale":"2024-09-15T21:01:40.552253+0000","last_undegraded":"2024-09-15T21:01:33.630775+0000","last_fullsized":"2024-09-15T21:01:33.628305+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":5,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":745472,"num_objects":4,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":7,"num_read_kb":1733,"num_write":25,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":3342336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,2147483647,3,2147483647,2147483647,2,1],"avail_no_missing":["0(3)","1(7)","3(0)","4(6)","7(2)","9(5)","10(4)","11(1)"],"object_location_counts":[{"shards":"0(0),1(7),2(6),3(3),5(1)","objects":3},{"shards":"0(3),1(7),3(0),4(6),7(2),9(5),10(4),11(1)","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"1"}]},{"pgid":"3.16","version":"105'44","reported_seq":41,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:01:40.567474+0000","last_change":"2024-09-15T21:01:34.334237+0000","last_active":"2024-09-15T21:01:40.567474+0000","last_peered":"2024-09-15T21:01:40.567474+0000","last_clean":"2024-09-15T21:01:31.471353+0000","last_became_active":"2024-09-15T21:01:34.224102+0000","last_became_peered":"2024-09-15T21:01:34.224102+0000","last_unstale":"2024-09-15T21:01:40.567474+0000","last_undegraded":"2024-09-15T21:01:33.684006+0000","last_fullsized":"2024-09-15T21:01:33.636792+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002370137,"stat_sum":{"num_bytes":1712128,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":6,"num_objects_missing":6,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":13,"num_read_kb":1915,"num_write":30,"num_write_kb":1832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2859008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,3,8,2,10,6],"acting":[5,2147483647,4,3,2147483647,2,2147483647,6],"avail_no_missing":["0(0)","4(2)","5(7)","7(3)","8(1)","9(6)","10(5)","11(4)"],"object_location_counts":[{"shards":"0(0),4(2),5(7),7(3),8(1),9(6),10(5),11(4)","objects":3},{"shards":"2(5),3(3),4(2),5(0),6(7)","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"2"},{"start":"20","length":"1"}]},{"pgid":"3.15","version":"107'222","reported_seq":62,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded","last_fresh":"2024-09-15T21:01:41.548124+0000","last_change":"2024-09-15T21:01:34.224389+0000","last_active":"2024-09-15T21:01:41.548124+0000","last_peered":"2024-09-15T21:01:41.548124+0000","last_clean":"2024-09-15T21:01:28.092733+0000","last_became_active":"2024-09-15T21:01:34.209161+0000","last_became_peered":"2024-09-15T21:01:34.209161+0000","last_unstale":"2024-09-15T21:01:41.548124+0000","last_undegraded":"2024-09-15T21:01:33.755887+0000","last_fullsized":"2024-09-15T21:01:33.742021+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21315584,"num_objects":11,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":13,"num_objects_missing":13,"num_objects_degraded":104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":36,"num_read_kb":3253,"num_write":90,"num_write_kb":24112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":15597568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[2147483647,2147483647,7,4,2,6,0,2147483647],"avail_no_missing":["2(2)","3(1)","5(0)","6(3)","7(4)","8(5)","10(7)","11(6)"],"object_location_counts":[{"shards":"0(6),2(4),4(3),6(5),7(2)","objects":5},{"shards":"2(2),3(1),5(0),6(3),7(4),8(5),10(7),11(6)","objects":6}],"blocked_by":[],"up_primary":8,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.14","version":"102'181","reported_seq":133,"reported_epoch":107,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:01:40.661513+0000","last_change":"2024-09-15T21:01:37.760200+0000","last_active":"2024-09-15T21:01:40.661513+0000","last_peered":"2024-09-15T21:01:40.661513+0000","last_clean":"2024-09-15T21:01:26.982743+0000","last_became_active":"2024-09-15T21:01:33.916378+0000","last_became_peered":"2024-09-15T21:01:33.916378+0000","last_unstale":"2024-09-15T21:01:40.661513+0000","last_undegraded":"2024-09-15T21:01:40.661513+0000","last_fullsized":"2024-09-15T21:01:33.630920+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":8,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":21,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":28,"num_read_kb":19584,"num_write":47,"num_write_kb":18616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":7127040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,2147483647,7,2147483647,2,5],"avail_no_missing":["0(7)","2(5)","5(3)","6(6)","7(4)","9(2)","10(0)","11(1)"],"object_location_counts":[{"shards":"0(7),2(5),5(3),6(6),7(4),9(2),10(0),11(1)","objects":3},{"shards":"2(6),3(2),4(0),5(7),7(4),8(1)","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.564452+0000","last_change":"2024-09-15T21:00:08.464015+0000","last_active":"2024-09-15T21:01:40.564452+0000","last_peered":"2024-09-15T21:01:40.564452+0000","last_clean":"2024-09-15T21:01:40.564452+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:01:40.564452+0000","last_undegraded":"2024-09-15T21:01:40.564452+0000","last_fullsized":"2024-09-15T21:01:40.564452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:08.004258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'44","reported_seq":455,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.552230+0000","last_change":"2024-09-15T21:01:37.666878+0000","last_active":"2024-09-15T21:01:40.552230+0000","last_peered":"2024-09-15T21:01:40.552230+0000","last_clean":"2024-09-15T21:01:40.552230+0000","last_became_active":"2024-09-15T21:01:30.958712+0000","last_became_peered":"2024-09-15T21:01:30.958712+0000","last_unstale":"2024-09-15T21:01:40.552230+0000","last_undegraded":"2024-09-15T21:01:40.552230+0000","last_fullsized":"2024-09-15T21:01:40.552230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'41","last_scrub_stamp":"2024-09-15T21:01:37.339216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.339216+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:40.058424+0000","scrub_duration":723,"objects_trimmed":0,"snaptrim_duration":0.004198416,"stat_sum":{"num_bytes":327680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4,7,11,10,9,5],"acting":[0,8,4,7,11,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":127,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.556030+0000","last_change":"2024-09-15T21:00:42.352742+0000","last_active":"2024-09-15T21:01:40.556030+0000","last_peered":"2024-09-15T21:01:40.556030+0000","last_clean":"2024-09-15T21:01:40.556030+0000","last_became_active":"2024-09-15T21:00:42.352075+0000","last_became_peered":"2024-09-15T21:00:42.352075+0000","last_unstale":"2024-09-15T21:01:40.556030+0000","last_undegraded":"2024-09-15T21:01:40.556030+0000","last_fullsized":"2024-09-15T21:01:40.556030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:32.658647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3],"acting":[8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"95'11","reported_seq":426,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.558024+0000","last_change":"2024-09-15T21:01:37.673352+0000","last_active":"2024-09-15T21:01:40.558024+0000","last_peered":"2024-09-15T21:01:40.558024+0000","last_clean":"2024-09-15T21:01:40.558024+0000","last_became_active":"2024-09-15T21:01:31.161568+0000","last_became_peered":"2024-09-15T21:01:31.161568+0000","last_unstale":"2024-09-15T21:01:40.558024+0000","last_undegraded":"2024-09-15T21:01:40.558024+0000","last_fullsized":"2024-09-15T21:01:40.558024+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'11","last_scrub_stamp":"2024-09-15T21:01:35.352007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.352007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:47.669895+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0.00090138299999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":193,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.589102+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:01:40.589102+0000","last_peered":"2024-09-15T21:01:40.589102+0000","last_clean":"2024-09-15T21:01:40.589102+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:01:40.589102+0000","last_undegraded":"2024-09-15T21:01:40.589102+0000","last_fullsized":"2024-09-15T21:01:40.589102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'186","reported_seq":572,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.785357+0000","last_change":"2024-09-15T21:01:38.390932+0000","last_active":"2024-09-15T21:01:42.785357+0000","last_peered":"2024-09-15T21:01:42.785357+0000","last_clean":"2024-09-15T21:01:42.785357+0000","last_became_active":"2024-09-15T21:01:31.064590+0000","last_became_peered":"2024-09-15T21:01:31.064590+0000","last_unstale":"2024-09-15T21:01:42.785357+0000","last_undegraded":"2024-09-15T21:01:42.785357+0000","last_fullsized":"2024-09-15T21:01:42.785357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'179","last_scrub_stamp":"2024-09-15T21:01:35.859573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.859573+0000","objects_scrubbed":6,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.557103+0000","scrub_duration":355,"objects_trimmed":0,"snaptrim_duration":0.70725242700000002,"stat_sum":{"num_bytes":15695872,"num_objects":7,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,11,9,5,7,2,6,0],"acting":[10,11,9,5,7,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.551737+0000","last_change":"2024-09-15T21:00:42.350121+0000","last_active":"2024-09-15T21:01:40.551737+0000","last_peered":"2024-09-15T21:01:40.551737+0000","last_clean":"2024-09-15T21:01:40.551737+0000","last_became_active":"2024-09-15T21:00:42.349833+0000","last_became_peered":"2024-09-15T21:00:42.349833+0000","last_unstale":"2024-09-15T21:01:40.551737+0000","last_undegraded":"2024-09-15T21:01:40.551737+0000","last_fullsized":"2024-09-15T21:01:40.551737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:10.241744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10],"acting":[0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"106'236","reported_seq":679,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567882+0000","last_change":"2024-09-15T21:01:38.431997+0000","last_active":"2024-09-15T21:01:40.567882+0000","last_peered":"2024-09-15T21:01:40.567882+0000","last_clean":"2024-09-15T21:01:40.567882+0000","last_became_active":"2024-09-15T21:01:30.973822+0000","last_became_peered":"2024-09-15T21:01:30.973822+0000","last_unstale":"2024-09-15T21:01:40.567882+0000","last_undegraded":"2024-09-15T21:01:40.567882+0000","last_fullsized":"2024-09-15T21:01:40.567882+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'219","last_scrub_stamp":"2024-09-15T21:01:35.157208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.157208+0000","objects_scrubbed":9,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:55.619851+0000","scrub_duration":211,"objects_trimmed":0,"snaptrim_duration":0.65588598099999995,"stat_sum":{"num_bytes":18317312,"num_objects":12,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":24,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,8,11,10],"acting":[5,3,2,6,7,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.3","version":"91'46","reported_seq":426,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.562694+0000","last_change":"2024-09-15T21:01:37.767656+0000","last_active":"2024-09-15T21:01:40.562694+0000","last_peered":"2024-09-15T21:01:40.562694+0000","last_clean":"2024-09-15T21:01:40.562694+0000","last_became_active":"2024-09-15T21:01:31.264196+0000","last_became_peered":"2024-09-15T21:01:31.264196+0000","last_unstale":"2024-09-15T21:01:40.562694+0000","last_undegraded":"2024-09-15T21:01:40.562694+0000","last_fullsized":"2024-09-15T21:01:40.562694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'46","last_scrub_stamp":"2024-09-15T21:01:35.068798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.068798+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:03.665957+0000","scrub_duration":371,"objects_trimmed":0,"snaptrim_duration":0.0017916,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.2","version":"24'2","reported_seq":212,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567156+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:01:40.567156+0000","last_peered":"2024-09-15T21:01:40.567156+0000","last_clean":"2024-09-15T21:01:40.567156+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:01:40.567156+0000","last_undegraded":"2024-09-15T21:01:40.567156+0000","last_fullsized":"2024-09-15T21:01:40.567156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"104'73","reported_seq":499,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.885290+0000","last_change":"2024-09-15T21:01:38.790030+0000","last_active":"2024-09-15T21:01:42.885290+0000","last_peered":"2024-09-15T21:01:42.885290+0000","last_clean":"2024-09-15T21:01:42.885290+0000","last_became_active":"2024-09-15T21:01:31.363252+0000","last_became_peered":"2024-09-15T21:01:31.363252+0000","last_unstale":"2024-09-15T21:01:42.885290+0000","last_undegraded":"2024-09-15T21:01:42.885290+0000","last_fullsized":"2024-09-15T21:01:42.885290+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'69","last_scrub_stamp":"2024-09-15T21:01:38.158232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:38.158232+0000","objects_scrubbed":8,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:04.945407+0000","scrub_duration":1771,"objects_trimmed":0,"snaptrim_duration":0.13175252800000001,"stat_sum":{"num_bytes":15351808,"num_objects":8,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.786750+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:01:40.786750+0000","last_peered":"2024-09-15T21:01:40.786750+0000","last_clean":"2024-09-15T21:01:40.786750+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:01:40.786750+0000","last_undegraded":"2024-09-15T21:01:40.786750+0000","last_fullsized":"2024-09-15T21:01:40.786750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"104'60","reported_seq":501,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.651928+0000","last_change":"2024-09-15T21:01:37.763625+0000","last_active":"2024-09-15T21:01:40.651928+0000","last_peered":"2024-09-15T21:01:40.651928+0000","last_clean":"2024-09-15T21:01:40.651928+0000","last_became_active":"2024-09-15T21:01:31.056403+0000","last_became_peered":"2024-09-15T21:01:31.056403+0000","last_unstale":"2024-09-15T21:01:40.651928+0000","last_undegraded":"2024-09-15T21:01:40.651928+0000","last_fullsized":"2024-09-15T21:01:40.651928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'55","last_scrub_stamp":"2024-09-15T21:01:36.098483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.098483+0000","objects_scrubbed":2,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:38.253595+0000","scrub_duration":459,"objects_trimmed":0,"snaptrim_duration":0.001132278,"stat_sum":{"num_bytes":3899392,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":185,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.693150+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:01:40.693150+0000","last_peered":"2024-09-15T21:01:40.693150+0000","last_clean":"2024-09-15T21:01:40.693150+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:01:40.693150+0000","last_undegraded":"2024-09-15T21:01:40.693150+0000","last_fullsized":"2024-09-15T21:01:40.693150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"104'93","reported_seq":579,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.658064+0000","last_change":"2024-09-15T21:01:39.194412+0000","last_active":"2024-09-15T21:01:40.658064+0000","last_peered":"2024-09-15T21:01:40.658064+0000","last_clean":"2024-09-15T21:01:40.658064+0000","last_became_active":"2024-09-15T21:01:31.166834+0000","last_became_peered":"2024-09-15T21:01:31.166834+0000","last_unstale":"2024-09-15T21:01:40.658064+0000","last_undegraded":"2024-09-15T21:01:40.658064+0000","last_fullsized":"2024-09-15T21:01:40.658064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"98'85","last_scrub_stamp":"2024-09-15T21:01:36.874673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.874673+0000","objects_scrubbed":7,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:00.638345+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0.60580713200000003,"stat_sum":{"num_bytes":9224192,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"2.3","version":"22'1","reported_seq":187,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.567360+0000","last_change":"2024-09-15T21:00:07.905687+0000","last_active":"2024-09-15T21:01:40.567360+0000","last_peered":"2024-09-15T21:01:40.567360+0000","last_clean":"2024-09-15T21:01:40.567360+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:01:40.567360+0000","last_undegraded":"2024-09-15T21:01:40.567360+0000","last_fullsized":"2024-09-15T21:01:40.567360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":307,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.693773+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:01:40.693773+0000","last_peered":"2024-09-15T21:01:40.693773+0000","last_clean":"2024-09-15T21:01:40.693773+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:01:40.693773+0000","last_undegraded":"2024-09-15T21:01:40.693773+0000","last_fullsized":"2024-09-15T21:01:40.693773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"104'28","reported_seq":424,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.784773+0000","last_change":"2024-09-15T21:01:38.662631+0000","last_active":"2024-09-15T21:01:42.784773+0000","last_peered":"2024-09-15T21:01:42.784773+0000","last_clean":"2024-09-15T21:01:42.784773+0000","last_became_active":"2024-09-15T21:01:31.267688+0000","last_became_peered":"2024-09-15T21:01:31.267688+0000","last_unstale":"2024-09-15T21:01:42.784773+0000","last_undegraded":"2024-09-15T21:01:42.784773+0000","last_fullsized":"2024-09-15T21:01:42.784773+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"97'24","last_scrub_stamp":"2024-09-15T21:01:36.086556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:36.086556+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:51.493292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0.096872859000000006,"stat_sum":{"num_bytes":5537792,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":12437,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,7,2,4,3,5,10,8],"acting":[11,7,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.9","version":"103'79","reported_seq":529,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.555268+0000","last_change":"2024-09-15T21:01:37.894900+0000","last_active":"2024-09-15T21:01:40.555268+0000","last_peered":"2024-09-15T21:01:40.555268+0000","last_clean":"2024-09-15T21:01:40.555268+0000","last_became_active":"2024-09-15T21:01:30.773553+0000","last_became_peered":"2024-09-15T21:01:30.773553+0000","last_unstale":"2024-09-15T21:01:40.555268+0000","last_undegraded":"2024-09-15T21:01:40.555268+0000","last_fullsized":"2024-09-15T21:01:40.555268+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'77","last_scrub_stamp":"2024-09-15T21:01:35.180901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:35.180901+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.710717+0000","scrub_duration":503,"objects_trimmed":0,"snaptrim_duration":0.101288825,"stat_sum":{"num_bytes":4489216,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.a","version":"103'17","reported_seq":389,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.562313+0000","last_change":"2024-09-15T21:01:37.778516+0000","last_active":"2024-09-15T21:01:40.562313+0000","last_peered":"2024-09-15T21:01:40.562313+0000","last_clean":"2024-09-15T21:01:40.562313+0000","last_became_active":"2024-09-15T21:00:51.178157+0000","last_became_peered":"2024-09-15T21:00:51.178157+0000","last_unstale":"2024-09-15T21:01:40.562313+0000","last_undegraded":"2024-09-15T21:01:40.562313+0000","last_fullsized":"2024-09-15T21:01:40.562313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:01:46.196972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040306170000000002,"stat_sum":{"num_bytes":4423680,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":12576,"num_write":12,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5,10,8,9,3,7],"acting":[6,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.b","version":"104'102","reported_seq":501,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.558307+0000","last_change":"2024-09-15T21:01:38.467385+0000","last_active":"2024-09-15T21:01:40.558307+0000","last_peered":"2024-09-15T21:01:40.558307+0000","last_clean":"2024-09-15T21:01:40.558307+0000","last_became_active":"2024-09-15T21:00:51.891415+0000","last_became_peered":"2024-09-15T21:00:51.891415+0000","last_unstale":"2024-09-15T21:01:40.558307+0000","last_undegraded":"2024-09-15T21:01:40.558307+0000","last_fullsized":"2024-09-15T21:01:40.558307+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69506695900000004,"stat_sum":{"num_bytes":12288000,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":29767,"num_write":48,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":7159808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,6,9],"acting":[3,0,4,8,11,7,6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.c","version":"104'98","reported_seq":446,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.667323+0000","last_change":"2024-09-15T21:01:37.988256+0000","last_active":"2024-09-15T21:01:40.667323+0000","last_peered":"2024-09-15T21:01:40.667323+0000","last_clean":"2024-09-15T21:01:40.667323+0000","last_became_active":"2024-09-15T21:00:20.390220+0000","last_became_peered":"2024-09-15T21:00:20.390220+0000","last_unstale":"2024-09-15T21:01:40.667323+0000","last_undegraded":"2024-09-15T21:01:40.667323+0000","last_fullsized":"2024-09-15T21:01:40.667323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31138399500000002,"stat_sum":{"num_bytes":10354688,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":11641,"num_write":55,"num_write_kb":10656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.d","version":"104'31","reported_seq":390,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.692768+0000","last_change":"2024-09-15T21:01:38.883235+0000","last_active":"2024-09-15T21:01:40.692768+0000","last_peered":"2024-09-15T21:01:40.692768+0000","last_clean":"2024-09-15T21:01:40.692768+0000","last_became_active":"2024-09-15T21:00:20.321901+0000","last_became_peered":"2024-09-15T21:00:20.321901+0000","last_unstale":"2024-09-15T21:01:40.692768+0000","last_undegraded":"2024-09-15T21:01:40.692768+0000","last_fullsized":"2024-09-15T21:01:40.692768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.118787121,"stat_sum":{"num_bytes":1687552,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":16,"num_read_kb":4922,"num_write":30,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.e","version":"104'22","reported_seq":422,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.786982+0000","last_change":"2024-09-15T21:01:37.696525+0000","last_active":"2024-09-15T21:01:42.786982+0000","last_peered":"2024-09-15T21:01:42.786982+0000","last_clean":"2024-09-15T21:01:42.786982+0000","last_became_active":"2024-09-15T21:01:25.767274+0000","last_became_peered":"2024-09-15T21:01:25.767274+0000","last_unstale":"2024-09-15T21:01:42.786982+0000","last_undegraded":"2024-09-15T21:01:42.786982+0000","last_fullsized":"2024-09-15T21:01:42.786982+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013156518000000001,"stat_sum":{"num_bytes":884736,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1258,"num_write":10,"num_write_kb":1952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":2375680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":348,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.694486+0000","last_change":"2024-09-15T21:01:37.790590+0000","last_active":"2024-09-15T21:01:40.694486+0000","last_peered":"2024-09-15T21:01:40.694486+0000","last_clean":"2024-09-15T21:01:40.694486+0000","last_became_active":"2024-09-15T21:00:20.390499+0000","last_became_peered":"2024-09-15T21:00:20.390499+0000","last_unstale":"2024-09-15T21:01:40.694486+0000","last_undegraded":"2024-09-15T21:01:40.694486+0000","last_fullsized":"2024-09-15T21:01:40.694486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012251160000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.10","version":"93'56","reported_seq":465,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-15T21:01:40.561209+0000","last_change":"2024-09-15T21:01:37.776011+0000","last_active":"2024-09-15T21:01:40.561209+0000","last_peered":"2024-09-15T21:01:40.561209+0000","last_clean":"2024-09-15T21:01:40.561209+0000","last_became_active":"2024-09-15T21:01:35.994337+0000","last_became_peered":"2024-09-15T21:01:35.994337+0000","last_unstale":"2024-09-15T21:01:40.561209+0000","last_undegraded":"2024-09-15T21:01:40.561209+0000","last_fullsized":"2024-09-15T21:01:40.561209+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":5,"last_scrub":"93'56","last_scrub_stamp":"2024-09-15T21:01:37.452341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:01:37.452341+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:41.786020+0000","scrub_duration":339,"objects_trimmed":0,"snaptrim_duration":0.002419652,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,10,7],"acting":[6,9,5,3,0,2,10,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.11","version":"107'57","reported_seq":460,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T21:01:42.787199+0000","last_change":"2024-09-15T21:01:42.467713+0000","last_active":"2024-09-15T21:01:42.787199+0000","last_peered":"2024-09-15T21:01:42.787199+0000","last_clean":"2024-09-15T21:01:42.787199+0000","last_became_active":"2024-09-15T21:01:40.966359+0000","last_became_peered":"2024-09-15T21:01:40.966359+0000","last_unstale":"2024-09-15T21:01:42.787199+0000","last_undegraded":"2024-09-15T21:01:42.787199+0000","last_fullsized":"2024-09-15T21:01:42.787199+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-15T21:01:51.490569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.398133026,"stat_sum":{"num_bytes":4120576,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":18894,"num_write":24,"num_write_kb":4088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":6905856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,8,11],"acting":[10,4,6,0,3,9,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"},{"start":"1c","length":"3"},{"start":"20","length":"1"}]},{"pgid":"3.12","version":"104'88","reported_seq":59,"reported_epoch":107,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:01:40.651707+0000","last_change":"2024-09-15T21:01:40.332812+0000","last_active":"2024-09-15T21:01:40.651707+0000","last_peered":"2024-09-15T21:01:40.651707+0000","last_clean":"2024-09-15T21:01:25.370146+0000","last_became_active":"2024-09-15T21:01:34.052109+0000","last_became_peered":"2024-09-15T21:01:34.052109+0000","last_unstale":"2024-09-15T21:01:40.651707+0000","last_undegraded":"2024-09-15T21:01:40.651707+0000","last_fullsized":"2024-09-15T21:01:33.630656+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10862592,"num_objects":7,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":16,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":22,"num_read_kb":9511,"num_write":37,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":16113664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,8,2,5,6,11],"acting":[0,7,3,2147483647,2,5,6,2147483647],"avail_no_missing":["1(5)","3(0)","4(3)","5(4)","6(2)","8(6)","9(1)","10(7)"],"object_location_counts":[{"shards":"0(0),2(4),3(2),5(5),6(6),7(1)","objects":5},{"shards":"1(5),3(0),4(3),5(4),6(2),8(6),9(1),10(7)","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]},{"pgid":"3.13","version":"91'46","reported_seq":37,"reported_epoch":107,"state":"active+recovery_wait+undersized+degraded","last_fresh":"2024-09-15T21:01:40.567157+0000","last_change":"2024-09-15T21:01:34.052850+0000","last_active":"2024-09-15T21:01:40.567157+0000","last_peered":"2024-09-15T21:01:40.567157+0000","last_clean":"2024-09-15T21:01:24.965153+0000","last_became_active":"2024-09-15T21:01:34.045247+0000","last_became_peered":"2024-09-15T21:01:34.045247+0000","last_unstale":"2024-09-15T21:01:40.567157+0000","last_undegraded":"2024-09-15T21:01:33.682859+0000","last_fullsized":"2024-09-15T21:01:33.634429+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:18.983923+0000","last_clean_scrub_stamp":"2024-09-15T21:00:18.983923+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6094848,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":10,"num_objects_missing":10,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8,"num_read_kb":1159,"num_write":19,"num_write_kb":6504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[2147483647,5,4,2,6,1,9,2147483647],"avail_no_missing":["0(2)","3(1)","4(0)","5(5)","6(3)","9(6)","10(7)","11(4)"],"object_location_counts":[{"shards":"0(2),3(1),4(0),5(5),6(3),9(6),10(7),11(4)","objects":6},{"shards":"1(5),2(3),4(2),5(1),6(4),9(6)","objects":-2}],"blocked_by":[],"up_primary":7,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1"},{"start":"20","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":171196416,"num_objects":107,"num_object_clones":63,"num_object_copies":856,"num_objects_missing_on_primary":45,"num_objects_missing":45,"num_objects_degraded":303,"num_objects_misplaced":37,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":7,"num_read":253,"num_read_kb":130287,"num_write":519,"num_write_kb":112384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":78266368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454422528,"data_stored":683126784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1939,"ondisk_log_size":1939,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65876,"kb_used_data":34692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94305964,"statfs":{"total":96636764160,"available":96569307136,"internally_reserved":0,"allocated":35524608,"data_stored":53076607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411353,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105788,"kb_used_data":52396,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94266052,"statfs":{"total":96636764160,"available":96528437248,"internally_reserved":0,"allocated":53653504,"data_stored":78918815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411352,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93568,"kb_used_data":40172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94278272,"statfs":{"total":96636764160,"available":96540950528,"internally_reserved":0,"allocated":41136128,"data_stored":62173823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411351,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68340,"kb_used_data":37156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303500,"statfs":{"total":96636764160,"available":96566784000,"internally_reserved":0,"allocated":38047744,"data_stored":55534207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411351,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69860,"kb_used_data":38676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94301980,"statfs":{"total":96636764160,"available":96565227520,"internally_reserved":0,"allocated":39604224,"data_stored":60691615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444056,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68340,"kb_used_data":37144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303500,"statfs":{"total":96636764160,"available":96566784000,"internally_reserved":0,"allocated":38035456,"data_stored":54039167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98804,"kb_used_data":49560,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94273036,"statfs":{"total":96636764160,"available":96535588864,"internally_reserved":0,"allocated":50749440,"data_stored":74908287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444056,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101688,"kb_used_data":48292,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94270152,"statfs":{"total":96636764160,"available":96532635648,"internally_reserved":0,"allocated":49451008,"data_stored":74445458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444055,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":28784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":29474816,"data_stored":40120959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,0,2,5,8,2,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444057,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68320,"kb_used_data":37136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303520,"statfs":{"total":96636764160,"available":96566804480,"internally_reserved":0,"allocated":38027264,"data_stored":56660607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444055,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47680,"kb_used_data":16484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324160,"statfs":{"total":96636764160,"available":96587939840,"internally_reserved":0,"allocated":16879616,"data_stored":20947602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444056,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63040,"kb_used_data":31844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308800,"statfs":{"total":96636764160,"available":96572211200,"internally_reserved":0,"allocated":32608256,"data_stored":45744767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,3,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31858688,"data_stored":46567424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15572992,"data_stored":20144128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34631680,"data_stored":52682752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36724736,"data_stored":55857152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29749248,"data_stored":41967616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":48439296,"data_stored":73936896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49979392,"data_stored":74645504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":38576128,"data_stored":60166144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":38404096,"data_stored":57462784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36761600,"data_stored":54337536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":40038400,"data_stored":61575168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":53686272,"data_stored":83783680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:01:46.673 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-15T21:01:46.673 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-15T21:01:46.674 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5: {} 2024-09-15T21:01:46.674 INFO:teuthology.orchestra.run.smithi027.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:01:46.681 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:87 3.1s0 3:81f89603:::smithi02732947-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:46.682 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.412105573000005, 2024-09-15T21:01:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59750891500000003, 2024-09-15T21:01:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:46.683 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 87 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.684 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264907+0000", 2024-09-15T21:01:46.685 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264908+0000", 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.63e-07 2024-09-15T21:01:46.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264910+0000", 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.204e-06 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.687 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264916+0000", 2024-09-15T21:01:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3949999999999998e-06 2024-09-15T21:01:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.688 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.264921+0000", 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5850000000000001e-06 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.689 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367481+0000", 2024-09-15T21:01:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10256029999999999 2024-09-15T21:01:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:46.690 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.367582+0000", 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000100735 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663504+0000", 2024-09-15T21:01:46.691 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29592205999999999 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.663746+0000", 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024177399999999999 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.692 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.862416+0000", 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.198670399 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.693 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:256 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~0] snapc 4=[3] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 75.451597339000003, 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40446953000000002, 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.694 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 256 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.695 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:46.696 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225416+0000", 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7899999999999998e-07 2024-09-15T21:01:46.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225420+0000", 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3220000000000001e-06 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.698 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225430+0000", 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0073e-05 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.699 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.225436+0000", 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.246e-06 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227285+0000", 2024-09-15T21:01:46.700 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00184917 2024-09-15T21:01:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227351+0000", 2024-09-15T21:01:46.701 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5983999999999997e-05 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.227662+0000", 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000311212 2024-09-15T21:01:46.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.228139+0000", 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047669100000000001 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.703 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629764+0000", 2024-09-15T21:01:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40162525100000002 2024-09-15T21:01:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.704 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:31.629885+0000", 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000121002 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.705 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:354 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 8] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 66.534125739999993, 2024-09-15T21:01:46.706 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.41456612199999998, 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.707 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 354 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.708 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142887+0000", 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142889+0000", 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5430000000000001e-06 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142900+0000", 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1269e-05 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.711 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142923+0000", 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142948+0000", 2024-09-15T21:01:46.712 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4511999999999998e-05 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143223+0000", 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000275075 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.713 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143246+0000", 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3238999999999999e-05 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.714 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.151903+0000", 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0086569879999999991 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.715 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.152227+0000", 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00032339800000000002 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154103+0000", 2024-09-15T21:01:46.716 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001876544 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.157039+0000", 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0029360300000000001 2024-09-15T21:01:46.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557389+0000", 2024-09-15T21:01:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40034982099999999 2024-09-15T21:01:46.718 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557453+0000", 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4480999999999995e-05 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.719 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:355 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:46.720 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 66.534034352999996, 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.414600568, 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 355 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.722 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142979+0000", 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142980+0000", 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5099999999999999e-06 2024-09-15T21:01:46.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.142987+0000", 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.5560000000000004e-06 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.724 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143007+0000", 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0126e-05 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143022+0000", 2024-09-15T21:01:46.725 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5608999999999999e-05 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143361+0000", 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033901800000000001 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.726 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.143381+0000", 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9295000000000001e-05 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.727 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.154994+0000", 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.011613573 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.728 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.155204+0000", 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000209423 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.156480+0000", 2024-09-15T21:01:46.729 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0012763130000000001 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.160406+0000", 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0039257959999999996 2024-09-15T21:01:46.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557538+0000", 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39713242999999998 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.557579+0000", 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0918999999999999e-05 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:360 3.6s0 3:6f7bd2df:::smithi02732947-21:head [append 0~507904] snapc 9=[9,8] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 66.370337192999997, 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44936693500000002, 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.733 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 360 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306676+0000", 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.734 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306677+0000", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5099999999999999e-07 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306970+0000", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029313599999999999 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.306981+0000", 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1053000000000001e-05 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407077+0000", 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096407 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407172+0000", 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4603000000000002e-05 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.736 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407331+0000", 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000158896 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.408673+0000", 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001342623 2024-09-15T21:01:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413632+0000", 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0049585439999999996 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.755971+0000", 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34233954300000002 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.756043+0000", 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1378999999999995e-05 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:363 3.6s0 3:6f7bd2df:::smithi02732947-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 66.269480430000002, 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.457850433, 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 363 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407533+0000", 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.5000000000000001e-07 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407539+0000", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.383e-06 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407551+0000", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1786e-05 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.407559+0000", 2024-09-15T21:01:46.742 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4160000000000004e-06 2024-09-15T21:01:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413484+0000", 2024-09-15T21:01:46.743 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0059251140000000004 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.413617+0000", 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013288800000000001 2024-09-15T21:01:46.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760137+0000", 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.34651994899999999 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.760251+0000", 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00011369 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.865383+0000", 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105132357 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.746 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:772 3.1s0 3:81f89603:::smithi02732947-29:head [copy-from ver 76] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:46.747 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.042347476, 2024-09-15T21:01:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.64200396500000001, 2024-09-15T21:01:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.748 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 772 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.750 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634666+0000", 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634669+0000", 2024-09-15T21:01:46.751 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4699999999999998e-06 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634686+0000", 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7204999999999999e-05 2024-09-15T21:01:46.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634729+0000", 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3251999999999999e-05 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.753 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634756+0000", 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6271e-05 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634873+0000", 2024-09-15T21:01:46.754 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000116989 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634882+0000", 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638177+0000", 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003295035 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.756 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638307+0000", 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013013599999999999 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.757 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.165965+0000", 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52765778900000004 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.171723+0000", 2024-09-15T21:01:46.758 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0057583169999999998 2024-09-15T21:01:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276516+0000", 2024-09-15T21:01:46.759 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.104793367 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.276669+0000", 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000153046 2024-09-15T21:01:46.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.761 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:773 3.1s0 3:81f89603:::smithi02732947-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 31.042204040000001, 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.53435976299999999, 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 773 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634809+0000", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634811+0000", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.804e-06 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634818+0000", 2024-09-15T21:01:46.764 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4549999999999998e-06 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634843+0000", 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4895000000000001e-05 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634866+0000", 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3198e-05 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.765 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634983+0000", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000117088 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.634995+0000", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1912e-05 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638554+0000", 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00355902 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.766 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:15.638636+0000", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.1588999999999994e-05 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169005+0000", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.530369117 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169073+0000", 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7940000000000003e-05 2024-09-15T21:01:46.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:16.169169+0000", 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.5744999999999999e-05 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:850 3.1s0 3:8d17a01b:::smithi02732947-8:head [rollback 1e] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 24.011375078, 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3063522750000001, 2024-09-15T21:01:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 850 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.769 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665638+0000", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665639+0000", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8500000000000001e-07 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665642+0000", 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999999e-06 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.665651+0000", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4620000000000002e-06 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765755+0000", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100104183 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766158+0000", 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00040273000000000002 2024-09-15T21:01:46.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766308+0000", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015033700000000001 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.766461+0000", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015210000000000001 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371273+0000", 2024-09-15T21:01:46.772 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6048122309999999 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371725+0000", 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00045230200000000002 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663049+0000", 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29132430799999998 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.773 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.663312+0000", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026254099999999998 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.665305+0000", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019926309999999999 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870050+0000", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20474537700000001 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.774 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971470+0000", 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10142055 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.971990+0000", 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00051970900000000003 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:851 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~0] snapc 1e=[1e,1d,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 23.91121493, 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3075131120000001, 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 851 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765798+0000", 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765800+0000", 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1660000000000001e-06 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.777 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765811+0000", 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0787e-05 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765844+0000", 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2492e-05 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.765866+0000", 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2135999999999999e-05 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.778 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769059+0000", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031937459999999999 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for degraded object", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:22.769218+0000", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015836100000000001 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.371923+0000", 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6027050859999998 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.779 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.372518+0000", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00059501300000000003 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666553+0000", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29403548499999999 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.666683+0000", 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129827 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.667532+0000", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084906999999999995 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:25.870061+0000", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20252906200000001 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.072834+0000", 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202773013 2024-09-15T21:01:46.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:26.073311+0000", 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047686799999999998 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:902 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~425984] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.078525920000001, 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50413778399999998, 2024-09-15T21:01:46.782 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 902 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.783 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598487+0000", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598489+0000", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.685e-06 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598759+0000", 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000270079 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598779+0000", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0349000000000001e-05 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.598790+0000", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0718e-05 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.681231+0000", 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.082440988000000007 2024-09-15T21:01:46.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for peered", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.681236+0000", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.587e-06 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.783400+0000", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10216322899999999 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.783592+0000", 2024-09-15T21:01:46.786 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019240800000000001 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.784623+0000", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010312710000000001 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.886847+0000", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.102223972 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.102542+0000", 2024-09-15T21:01:46.787 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.21569451100000001 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.102625+0000", 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2986999999999997e-05 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:903 3.1s0 3:8d17a01b:::smithi02732947-8:head [append 0~573440] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 19.071925665999999, 2024-09-15T21:01:46.788 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59821814399999995, 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 903 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.789 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605087+0000", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605090+0000", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3199999999999998e-06 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605629+0000", 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053903700000000002 2024-09-15T21:01:46.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605648+0000", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9008000000000002e-05 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.605658+0000", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0152999999999999e-05 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.887227+0000", 2024-09-15T21:01:46.791 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.281569399 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.887364+0000", 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000136565 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.888691+0000", 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001326785 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.792 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.892180+0000", 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0034895339999999999 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.203243+0000", 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.31106281699999999 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.203305+0000", 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2526000000000004e-05 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 94/93 ECSubWriteReply(tid=158, last_complete=94'53, committed=1, applied=1))", 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 18.677486782999999, 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40489114599999998, 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:46.794 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999526+0000", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999527+0000", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.850000000000001e-07 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999535+0000", 2024-09-15T21:01:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6650000000000003e-06 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999548+0000", 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3084e-05 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:27.999558+0000", 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0012e-05 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.796 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404398+0000", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40484052599999998 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:28.404417+0000", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8873999999999999e-05 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:934 3.12s0 3:48a8a569:::smithi02732947-39:head [append 0~458752] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:36.660988+0000", 2024-09-15T21:01:46.797 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.016024979000001, 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.270818346, 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 934 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.660988+0000", 2024-09-15T21:01:46.798 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.660988+0000", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.660991+0000", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1130000000000001e-06 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.661544+0000", 2024-09-15T21:01:46.799 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00055316899999999995 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.661578+0000", 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3942000000000001e-05 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.661596+0000", 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7393000000000001e-05 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.800 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.669920+0000", 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0083241730000000007 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "op must wait for map", 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.669976+0000", 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5695000000000001e-05 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.691273+0000", 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.021297861000000001 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.801 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for missing object", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695719+0000", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0044455120000000004 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.606614+0000", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.91089559200000003 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.607090+0000", 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047506700000000002 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.802 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.611144+0000", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0040541860000000004 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.619914+0000", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0087707410000000003 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.929861+0000", 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30994607800000001 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.803 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.931806+0000", 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019458240000000001 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:935 3.12s0 3:48a8a569:::smithi02732947-39:head [append 0~475136] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:36.668771+0000", 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.008241683, 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.477272286, 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 935 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.668771+0000", 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.805 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.668771+0000", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.668775+0000", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3160000000000001e-06 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.669558+0000", 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00078290300000000005 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.806 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.669601+0000", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3353000000000003e-05 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.669626+0000", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4783e-05 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.673867+0000", 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0042416900000000002 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.673966+0000", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9061000000000006e-05 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695792+0000", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.021825458999999998 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for missing object", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695968+0000", 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000176002 2024-09-15T21:01:46.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.613657+0000", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.91768926900000003 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.614571+0000", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00091374900000000005 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.617835+0000", 2024-09-15T21:01:46.809 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.003263756 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.620364+0000", 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0025296289999999998 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.144464+0000", 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.52410016500000001 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.810 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.146044+0000", 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001579151 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:936 3.12s0 3:48a8a569:::smithi02732947-39:head [append 0~655360] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:36.682322+0000", 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.9946908249999993, 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.465775864, 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.811 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 936 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.682322+0000", 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.812 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.682322+0000", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.682327+0000", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.8729999999999998e-06 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.683289+0000", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000962085 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.813 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.683320+0000", 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0873000000000001e-05 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.683351+0000", 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0626000000000001e-05 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.696020+0000", 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.012669732 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.814 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for missing object", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.696198+0000", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017746600000000001 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.620862+0000", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.92466426000000002 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.621304+0000", 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00044150599999999998 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.815 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.624432+0000", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031285530000000001 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.825773+0000", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20134044100000001 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.146556+0000", 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.32078318 2024-09-15T21:01:46.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.148098+0000", 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001542269 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:937 3.12s0 3:48a8a569:::smithi02732947-39:head [append 0~671744] snapc 22=[22,21,1f,1b,1a,16] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:36.694948+0000", 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 9.9820651219999998, 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5554717419999999, 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 937 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.694948+0000", 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.818 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.694948+0000", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.694950+0000", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4679999999999999e-06 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695479+0000", 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00052844800000000005 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695553+0000", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4486999999999993e-05 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.695586+0000", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2722000000000003e-05 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.696252+0000", 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066570699999999997 2024-09-15T21:01:46.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for missing object", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:36.696396+0000", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000143893 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.720170+0000", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.023774755 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.720455+0000", 2024-09-15T21:01:46.821 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000284818 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.722343+0000", 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001887634 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.825776+0000", 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10343347 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.822 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.248776+0000", 2024-09-15T21:01:46.823 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.42299923099999998 2024-09-15T21:01:46.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.823 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.250420+0000", 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0016441089999999999 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:977 3.17s0 3:ec95157b:::smithi02732947-28:head [stat,delete] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.721687+0000", 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 8.955326286, 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.43643727399999999, 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 977 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721687+0000", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.825 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721687+0000", 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721702+0000", 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4810000000000001e-05 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721720+0000", 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8912000000000001e-05 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.826 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721751+0000", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0870000000000001e-05 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.721773+0000", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1753e-05 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.823670+0000", 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101897179 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.827 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.824417+0000", 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00074652699999999998 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.158124+0000", 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.333707223 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:1000 3.6s0 3:6097bd07:::smithi02732947-11:head [copy-from ver 67] snapc 23=[23,22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-15T21:01:46.828 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:42.830492+0000", 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8465206730000001, 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65943679399999999, 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1000 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.829 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830492+0000", 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830492+0000", 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830495+0000", 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1799999999999999e-06 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830510+0000", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5434000000000002e-05 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830538+0000", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8039999999999999e-05 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830551+0000", 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3247999999999999e-05 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.831 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.831033+0000", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00048191800000000002 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.831242+0000", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020923200000000001 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.277050+0000", 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.44580746199999999 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.285273+0000", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0082228059999999992 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.489673+0000", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20439993300000001 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.489929+0000", 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000256541 2024-09-15T21:01:46.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:1001 3.6s0 3:6097bd07:::smithi02732947-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e107)", 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:42.830582+0000", 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 3.8464307230000001, 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.453385763, 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:46.834 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 1001 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830582+0000", 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830582+0000", 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.835 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830583+0000", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9999999999999996e-07 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830587+0000", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6229999999999999e-06 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830599+0000", 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2707999999999999e-05 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.830607+0000", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4139999999999997e-06 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.878679+0000", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.048072257 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:42.878735+0000", 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5565000000000001e-05 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.283259+0000", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40452448200000002 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.283598+0000", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000338783 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:43.283968+0000", 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000370131 2024-09-15T21:01:46.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:46.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:46.839 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:46.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:01:46.841 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6: {} 2024-09-15T21:01:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:01:46.841 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7: {} 2024-09-15T21:01:46.841 INFO:teuthology.orchestra.run.smithi027.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:01:46.842 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8: {} 2024-09-15T21:01:46.842 INFO:teuthology.orchestra.run.smithi027.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-15T21:01:46.842 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9: {} 2024-09-15T21:01:46.843 INFO:teuthology.orchestra.run.smithi027.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-15T21:01:46.843 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10: {} 2024-09-15T21:01:46.843 INFO:teuthology.orchestra.run.smithi027.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-15T21:01:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:01:46.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11: {} 2024-09-15T21:01:46.844 INFO:teuthology.orchestra.run.smithi027.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-15T21:01:46.845 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:46.835+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:46.924 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:46.919+0000 7fde79919640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:46.940 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:46.934+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:47.009 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:47.010 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:47.010 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:47.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:01:47.041 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:47.037+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:47.142 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-09-15T21:01:47.134+0000 7fbc22b20640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26248) UID: 0 2024-09-15T21:01:47.198 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:47.198 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:47.212 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:01:47.244 INFO:tasks.ceph.osd.6.smithi066.stderr:2024-09-15T21:01:47.239+0000 7fd5577fb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26396) UID: 0 2024-09-15T21:01:47.344 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-09-15T21:01:47.338+0000 7ff6debfd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26250) UID: 0 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=9, last_complete=32'7, committed=1, applied=1))", 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.616045858000007, 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30180569499999998, 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.378 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756906+0000", 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.53e-07 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756909+0000", 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5359999999999999e-06 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756917+0000", 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.816e-06 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.379 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756941+0000", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3954000000000001e-05 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058503+0000", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.301561721 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058711+0000", 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00020831499999999999 2024-09-15T21:01:47.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=10, last_complete=32'8, committed=1, applied=1))", 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.6159988, 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30190075700000002, 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.381 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.382 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756953+0000", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756956+0000", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9000000000000002e-06 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756965+0000", 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0790000000000007e-06 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.383 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756971+0000", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4030000000000004e-06 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058837+0000", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30186509099999997 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058853+0000", 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6918e-05 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=11, last_complete=32'9, committed=1, applied=1))", 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.615971000000002, 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30200892000000001, 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.385 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756981+0000", 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3299999999999998e-07 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.386 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756983+0000", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0870000000000001e-06 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756991+0000", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4449999999999996e-06 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.756997+0000", 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8769999999999998e-06 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.387 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058975+0000", 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30197816100000002 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.058989+0000", 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4017e-05 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 32/28 ECSubWriteReply(tid=12, last_complete=32'10, committed=1, applied=1))", 2024-09-15T21:01:47.388 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.615945605999997, 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30215523, 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.389 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757006+0000", 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2300000000000002e-07 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757009+0000", 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.367e-06 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.390 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757017+0000", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2789999999999998e-06 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.757023+0000", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3559999999999998e-06 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059146+0000", 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30212263499999997 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.059161+0000", 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5270000000000001e-05 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:106 3.4s0 3:235c4434:::smithi02732947-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.310566383999998, 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.51066560699999997, 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 106 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.393 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062385+0000", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062387+0000", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8899999999999999e-06 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062895+0000", 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00050759199999999996 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062913+0000", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8209000000000001e-05 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062926+0000", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3192e-05 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264285+0000", 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20135919999999999 2024-09-15T21:01:47.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.264668+0000", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038294700000000002 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.267279+0000", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0026112969999999998 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363244+0000", 2024-09-15T21:01:47.396 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095964646000000001 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572926+0000", 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.209681594 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.573051+0000", 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012504000000000001 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:107 3.4s0 3:235c4434:::smithi02732947-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.310005423999996, 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1179975739999999, 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 107 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.398 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062946+0000", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062947+0000", 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:47.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062954+0000", 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6889999999999998e-06 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.062966+0000", 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2044e-05 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.400 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163063+0000", 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100097125 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875847+0000", 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71278389799999997 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876001+0000", 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015423999999999999 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.401 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.876848+0000", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00084710500000000002 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883317+0000", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0064689860000000004 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180797+0000", 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29748011200000002 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.402 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.180944+0000", 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000146229 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:108 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.209847041000003, 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.224010174, 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 108 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.404 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163104+0000", 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163107+0000", 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5440000000000001e-06 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163120+0000", 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2575000000000001e-05 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.405 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163150+0000", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0389999999999999e-05 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163170+0000", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9788000000000001e-05 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877456+0000", 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71428638 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877705+0000", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00024871399999999998 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181077+0000", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30337212899999999 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181346+0000", 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026928999999999999 2024-09-15T21:01:47.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.387115+0000", 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20576836400000001 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:112 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.209750571000001, 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0198150960000001, 2024-09-15T21:01:47.408 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:47.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.410 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 112 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163201+0000", 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.163203+0000", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.694e-06 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164072+0000", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00086979199999999998 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164109+0000", 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6943000000000002e-05 2024-09-15T21:01:47.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164129+0000", 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9371000000000001e-05 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.877850+0000", 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71372171200000001 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.413 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.878240+0000", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038992999999999997 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.880206+0000", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019652039999999999 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883323+0000", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031173540000000001 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:47.414 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182786+0000", 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29946327700000003 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000229819 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:113 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:47.415 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.208793141000001, 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2197703820000001, 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 113 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:47.416 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164158+0000", 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164160+0000", 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.79e-06 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.417 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164169+0000", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4419999999999998e-06 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164191+0000", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2384000000000001e-05 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164205+0000", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4313e-05 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.418 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881232+0000", 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71702686299999996 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.881373+0000", 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000140445 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.882300+0000", 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092762699999999999 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.419 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883760+0000", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00146002 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383700+0000", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49993974699999999 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383929+0000", 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022875099999999999 2024-09-15T21:01:47.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:114 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.208723634999998, 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4253824179999999, 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:47.421 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 114 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.422 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164228+0000", 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164229+0000", 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.31e-06 2024-09-15T21:01:47.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164235+0000", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4729999999999999e-06 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164256+0000", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1124e-05 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.164274+0000", 2024-09-15T21:01:47.424 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8532000000000001e-05 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883114+0000", 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71883961399999996 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.883299+0000", 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018459999999999999 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.425 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384882+0000", 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.50158354199999999 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.385128+0000", 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000245672 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.589610+0000", 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20448255100000001 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=16, to_read={3:26d7dfe1:::smithi02732947-26:head=0,4096,0}, subchunks={3:26d7dfe1:::smithi02732947-26:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.108955929999993, 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30855138799999998, 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:47.427 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.428 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.263996+0000", 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363649+0000", 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099653380999999999 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.363664+0000", 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4667999999999999e-05 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.429 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572011+0000", 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.208347316 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572547+0000", 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00053602299999999995 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.005283757000001, 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30657130900000001, 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367668+0000", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367669+0000", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0729999999999999e-06 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367698+0000", 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8954000000000001e-05 2024-09-15T21:01:47.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367723+0000", 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4837e-05 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.433 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.367737+0000", 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4422e-05 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674072+0000", 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30633464999999999 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.674239+0000", 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016737300000000001 2024-09-15T21:01:47.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.992023986000007, 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494612673, 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.437 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380928+0000", 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380929+0000", 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7699999999999992e-07 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380951+0000", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2467e-05 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380968+0000", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.380982+0000", 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4201e-05 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875502+0000", 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.494519767 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875540+0000", 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8089000000000001e-05 2024-09-15T21:01:47.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.990083558999999, 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.49282801300000001, 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.441 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382868+0000", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382869+0000", 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3300000000000001e-07 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.442 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382890+0000", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1234e-05 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382913+0000", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2798000000000001e-05 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.382926+0000", 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3229e-05 2024-09-15T21:01:47.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875667+0000", 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.492740595 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875696+0000", 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9323999999999999e-05 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=16, attrs_read=0))", 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.800429932, 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30326375, 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572522+0000", 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572563+0000", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1526000000000002e-05 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.572579+0000", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5900999999999999e-05 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875757+0000", 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30317807299999999 2024-09-15T21:01:47.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.875785+0000", 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8249999999999999e-05 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.391419502999995, 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30173502400000002, 2024-09-15T21:01:47.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981532+0000", 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981534+0000", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.536e-06 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981544+0000", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0268999999999999e-05 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981573+0000", 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9677000000000001e-05 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.981593+0000", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9561000000000001e-05 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283234+0000", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30164134399999998 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.283267+0000", 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2636999999999999e-05 2024-09-15T21:01:47.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/28 ECSubWriteReply(tid=21, last_complete=33'14, committed=1, applied=1))", 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.383898095000006, 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.395688186, 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989053+0000", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989054+0000", 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.8299999999999995e-07 2024-09-15T21:01:47.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989061+0000", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1029999999999998e-06 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989081+0000", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0840999999999999e-05 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.989092+0000", 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0906e-05 2024-09-15T21:01:47.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.383575+0000", 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39448292800000001 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.384742+0000", 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001166425 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s0 33/28 ECSubRead(tid=22, to_read={3:235c4434:::smithi02732947-36:head=0,4096,0}, subchunks={3:235c4434:::smithi02732947-36:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.191018485000001, 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40658964800000003, 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181933+0000", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181955+0000", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.1997999999999998e-05 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.181964+0000", 2024-09-15T21:01:47.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7520000000000002e-06 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588157+0000", 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.406192889 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.588523+0000", 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00036600899999999998 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 33/28 ECSubReadReply(tid=22, attrs_read=0))", 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.190015407000004, 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30459709800000001, 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182936+0000", 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182937+0000", 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.294e-06 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.182978+0000", 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183002+0000", 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3907999999999999e-05 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.183016+0000", 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4591e-05 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487500+0000", 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30448341200000001 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.487533+0000", 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.3444e-05 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.463 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/28 ECSubReadReply(tid=60, attrs_read=0))", 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 72.525462602000005, 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30141633400000001, 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847489+0000", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847490+0000", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.4000000000000001e-07 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847501+0000", 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847511+0000", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.465e-06 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:34.847518+0000", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8909999999999998e-06 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148878+0000", 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30136025500000002 2024-09-15T21:01:47.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:35.148905+0000", 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7172999999999999e-05 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.467 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:47.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:01:47.469 INFO:tasks.ceph.osd.10.smithi092.stderr:2024-09-15T21:01:47.465+0000 7f8673373640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26613) UID: 0 2024-09-15T21:01:47.569 INFO:tasks.ceph.osd.7.smithi066.stderr:2024-09-15T21:01:47.563+0000 7f61ecd5c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26398) UID: 0 2024-09-15T21:01:47.650 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.651 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:47.651 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:47.656 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:47.651+0000 7fde7a91b640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-15T21:01:47.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:01:47.670 INFO:tasks.ceph.osd.9.smithi092.stderr:2024-09-15T21:01:47.665+0000 7fc95b668640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26612) UID: 0 2024-09-15T21:01:47.771 INFO:tasks.ceph.osd.5.smithi066.stderr:2024-09-15T21:01:47.767+0000 7fde95b31640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26394) UID: 0 2024-09-15T21:01:47.792 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.793 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:47.793 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:47.793 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:47.793 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:47.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:01:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'unique_pool_0' removed 2024-09-15T21:01:47.845 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T21:01:47.854 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T21:01:47.938 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:47.938 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:47.938 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:47.938 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/28 ECSubWrite(tid=23, reqid=client.4641.0:154, at_version=0'0, trim_to=0'0, roll_forward_to=33'3))", 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 82.340286000000006, 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20171277000000001, 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:47.939 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595551+0000", 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595553+0000", 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.940 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595577+0000", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3458000000000001e-05 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.595609+0000", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2289999999999997e-05 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.695767+0000", 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100158264 2024-09-15T21:01:47.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796046+0000", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100278822 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.796062+0000", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5634999999999999e-05 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797196+0000", 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0011338380000000001 2024-09-15T21:01:47.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.797264+0000", 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.847e-05 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.cs5 37/28 ECSubWrite(tid=63, reqid=client.4641.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=32'6))", 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 78.852333611000006, 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.19593087100000001, 2024-09-15T21:01:47.943 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083504+0000", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.0900000000000001e-07 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083519+0000", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5045000000000001e-05 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083534+0000", 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4712000000000001e-05 2024-09-15T21:01:47.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.083543+0000", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1230000000000006e-06 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179003+0000", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.095459523000000004 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.179013+0000", 2024-09-15T21:01:47.946 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0227999999999999e-05 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279398+0000", 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100385377 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:29.279435+0000", 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.6154e-05 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 38/28 ECSubWrite(tid=75, reqid=client.4641.0:238, at_version=0'0, trim_to=0'0, roll_forward_to=38'34))", 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 77.149319981000005, 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20232079, 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786517+0000", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786519+0000", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786546+0000", 2024-09-15T21:01:47.949 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6645e-05 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.786588+0000", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.2499000000000001e-05 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.886745+0000", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100156805 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987045+0000", 2024-09-15T21:01:47.950 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10029945699999999 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.987071+0000", 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.6058e-05 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988764+0000", 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00169301 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.951 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:30.988838+0000", 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.5s2 48/28 ECSubRead(tid=132, to_read={3:a5da2594:::smithi02732947-25:head=0,315392,0}, subchunks={3:a5da2594:::smithi02732947-25:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 67.472802497000004, 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20088298299999999, 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.953 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463035+0000", 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0800000000000004e-07 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463043+0000", 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.463058+0000", 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4732999999999999e-05 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.954 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.563142+0000", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008391699999999 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663346+0000", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100204423 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.663918+0000", 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00057127899999999995 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.955 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.064778941, 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30162102800000001, 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.956 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871058+0000", 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871059+0000", 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9999999999999995e-07 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.957 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871100+0000", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.0321999999999998e-05 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871128+0000", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8208000000000002e-05 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871152+0000", 2024-09-15T21:01:47.958 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4246000000000001e-05 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971464+0000", 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312292 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971482+0000", 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7176e-05 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.959 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172613+0000", 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20113102099999999 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.172679+0000", 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.6762999999999998e-05 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpRead(3.4s5 59/57 ECSubRead(tid=8, to_read={3:2b54c2df:::smithi02732947-23:head=102400,380928,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:47.960 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 57.064474081999997, 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20089554500000001, 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871363+0000", 2024-09-15T21:01:47.961 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871364+0000", 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2299999999999999e-07 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871378+0000", 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3706000000000001e-05 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.962 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.871408+0000", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0574999999999998e-05 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:50.971547+0000", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013905400000001 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072019+0000", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100471385 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.963 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.072259+0000", 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023990200000000001 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.036840570000003, 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20140428699999999, 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.964 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898997+0000", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.898998+0000", 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4980000000000001e-06 2024-09-15T21:01:47.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899107+0000", 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010874699999999999 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.899130+0000", 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.268e-05 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.966 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999226+0000", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100096764 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999366+0000", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000139231 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.999377+0000", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1097000000000001e-05 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.967 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100334+0000", 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100957065 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.100401+0000", 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.7205000000000005e-05 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 60/59 ECSubWrite(tid=180, reqid=client.4641.0:451, at_version=60'74, trim_to=0'0, roll_forward_to=0'0))", 2024-09-15T21:01:47.968 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 56.035062011000001, 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.200850169, 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.969 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900775+0000", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900776+0000", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900867+0000", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1712000000000006e-05 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.970 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:51.900880+0000", 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.218e-05 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001054+0000", 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10017451600000001 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001292+0000", 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00023806500000000001 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.971 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.001309+0000", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6680999999999998e-05 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101581+0000", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027267099999999 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:52.101625+0000", 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3965999999999999e-05 2024-09-15T21:01:47.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 68/53 ECSubWrite(tid=84, reqid=osd.11.0:84, at_version=68'11, trim_to=0'0, roll_forward_to=67'9))", 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 48.188061765999997, 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20168053699999999, 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.973 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747775+0000", 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747776+0000", 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8e-07 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.974 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747806+0000", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0182e-05 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.747817+0000", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0985000000000001e-05 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.847897+0000", 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100080024 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948088+0000", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100191302 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.948106+0000", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7869000000000001e-05 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949395+0000", 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001288909 2024-09-15T21:01:47.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:59.949456+0000", 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.1086000000000004e-05 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=236, reqid=osd.5.0:236, at_version=69'114, trim_to=0'0, roll_forward_to=68'112))", 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.884464893000001, 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.201432585, 2024-09-15T21:01:47.977 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051372+0000", 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051375+0000", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.8629999999999999e-06 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051452+0000", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6847000000000005e-05 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.051489+0000", 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.7295999999999997e-05 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151633+0000", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100143989 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151772+0000", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000138505 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.151792+0000", 2024-09-15T21:01:47.980 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0516000000000001e-05 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252713+0000", 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10092024300000001 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.252805+0000", 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2325999999999997e-05 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 69/59 ECSubWrite(tid=238, reqid=osd.5.0:237, at_version=0'0, trim_to=0'0, roll_forward_to=69'116))", 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.477361340000002, 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.202021808, 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458476+0000", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458478+0000", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2929999999999999e-06 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458507+0000", 2024-09-15T21:01:47.983 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9183000000000002e-05 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.458535+0000", 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7512999999999999e-05 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.558646+0000", 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100111397 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.984 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559059+0000", 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041233799999999999 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.559089+0000", 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0145999999999999e-05 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660438+0000", 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10134961000000001 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.985 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.660498+0000", 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9327999999999998e-05 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=56, reqid=client.4641.0:576, at_version=69'122, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.224910985999998, 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30235248100000001, 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710926+0000", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.710927+0000", 2024-09-15T21:01:47.987 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7000000000000001e-07 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711169+0000", 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000242449 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.711184+0000", 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4956999999999999e-05 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811338+0000", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10015429200000001 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911614+0000", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027530699999999 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.911632+0000", 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7963000000000001e-05 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.989 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.012893+0000", 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101261014 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013279+0000", 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038622899999999998 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 69/57 ECSubWrite(tid=57, reqid=client.4641.0:577, at_version=69'123, trim_to=0'0, roll_forward_to=69'119))", 2024-09-15T21:01:47.990 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 46.124444164000003, 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20197553200000001, 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811393+0000", 2024-09-15T21:01:47.991 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811394+0000", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1400000000000001e-06 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811565+0000", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017046300000000001 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811589+0000", 2024-09-15T21:01:47.992 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4229000000000001e-05 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.811601+0000", 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2194e-05 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912520+0000", 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100918439 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.993 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.912530+0000", 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0720000000000001e-05 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013321+0000", 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100790989 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:02.013369+0000", 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.7358000000000003e-05 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 75/59 ECSubWrite(tid=266, reqid=osd.5.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 41.168488943, 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.30105612199999998, 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:47.995 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767348+0000", 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767349+0000", 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.133e-06 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:47.996 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767364+0000", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4146e-05 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.767381+0000", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7144000000000001e-05 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.867460+0000", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100079691 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:47.997 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967708+0000", 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100247187 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:06.967719+0000", 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0937e-05 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068375+0000", 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100655967 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.998 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:07.068404+0000", 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9916999999999998e-05 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.5s2 78/59 ECSubWrite(tid=268, reqid=client.4641.0:643, at_version=78'141, trim_to=0'0, roll_forward_to=75'138))", 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.945510294999998, 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.203284137, 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:47.999 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990327+0000", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990328+0000", 2024-09-15T21:01:48.000 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990551+0000", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022275300000000001 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:09.990576+0000", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5414000000000001e-05 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.090713+0000", 2024-09-15T21:01:48.001 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10013612099999999 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.190988+0000", 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10027497 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.191018+0000", 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0602000000000003e-05 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.002 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193517+0000", 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0024985929999999999 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.193611+0000", 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.4324e-05 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=130, reqid=client.4641.0:832, at_version=87'14, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.184553057999999, 2024-09-15T21:01:48.003 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.188249429, 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751284+0000", 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751285+0000", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.0999999999999999e-07 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751828+0000", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00054304400000000001 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751839+0000", 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:48.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.751846+0000", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1110000000000002e-06 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836633+0000", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.084786880999999995 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.836646+0000", 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2927e-05 2024-09-15T21:01:48.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939450+0000", 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10280324 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939534+0000", 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.4116000000000006e-05 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=131, reqid=client.4641.0:833, at_version=87'15, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.178910000999998, 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.18273326100000001, 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756927+0000", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.756928+0000", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.9200000000000001e-07 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757088+0000", 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00015978499999999999 2024-09-15T21:01:48.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757098+0000", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0482999999999999e-05 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.757103+0000", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.2259999999999996e-06 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937980+0000", 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.180876436 2024-09-15T21:01:48.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.937987+0000", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.5440000000000001e-06 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939584+0000", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001597137 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.939661+0000", 2024-09-15T21:01:48.011 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.6057999999999996e-05 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 87/53 ECSubWrite(tid=132, reqid=client.4641.0:834, at_version=87'16, trim_to=0'0, roll_forward_to=79'12))", 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.174436275000001, 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.17882461699999999, 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761401+0000", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761402+0000", 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2500000000000005e-07 2024-09-15T21:01:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761528+0000", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012658 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761541+0000", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2294e-05 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.761547+0000", 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.117e-06 2024-09-15T21:01:48.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938455+0000", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.176908119 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.938472+0000", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6898999999999999e-05 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940190+0000", 2024-09-15T21:01:48.015 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001718687 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.940226+0000", 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.5196e-05 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.4s5 97/96 ECSubWrite(tid=125, reqid=osd.10.0:124, at_version=0'0, trim_to=0'0, roll_forward_to=97'179))", 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:31.369729+0000", 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 16.566108714999999, 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20161944600000001, 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.369729+0000", 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.369729+0000", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.369731+0000", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9599999999999999e-06 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.369754+0000", 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3051999999999999e-05 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.369797+0000", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.3359000000000003e-05 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.469912+0000", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100115289 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.470089+0000", 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00017691500000000001 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.470107+0000", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7439e-05 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.571243+0000", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101136091 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:31.571348+0000", 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00010534100000000001 2024-09-15T21:01:48.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.8s2 98/96 ECSubWrite(tid=172, reqid=osd.11.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=97'24))", 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:32.180205+0000", 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 15.7556318, 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20123785799999999, 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.021 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180205+0000", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180205+0000", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180206+0000", 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.3300000000000002e-07 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.022 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180228+0000", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2175999999999999e-05 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180283+0000", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.5281999999999997e-05 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.180307+0000", 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3682000000000001e-05 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.023 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.280620+0000", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100312583 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.280636+0000", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6575999999999999e-05 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.381301+0000", 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.100664584 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:32.381443+0000", 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000142442 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.025 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:48.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:01:48.159 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:48.159 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:48.159 INFO:teuthology.orchestra.run.smithi027.stdout: "num_ops": 0 2024-09-15T21:01:48.159 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:48.173 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:01:48.296 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:48.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [], 2024-09-15T21:01:48.296 INFO:teuthology.orchestra.run.smithi027.stdout: "complaint_time": 30, 2024-09-15T21:01:48.297 INFO:teuthology.orchestra.run.smithi027.stdout: "num_blocked_ops": 0 2024-09-15T21:01:48.297 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:48.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:01:48.442 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:01:48.442 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 20, 2024-09-15T21:01:48.442 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 600, 2024-09-15T21:01:48.442 INFO:teuthology.orchestra.run.smithi027.stdout: "ops": [ 2024-09-15T21:01:48.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:117 3.2s0 3:48a8a569:::smithi02732947-39:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 84.461188171000003, 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4189230779999999, 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 117 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.443 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976433+0000", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976434+0000", 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.3900000000000003e-07 2024-09-15T21:01:48.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976439+0000", 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9740000000000001e-06 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976450+0000", 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0998e-05 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.445 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.976459+0000", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2259999999999994e-06 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181560+0000", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.205100751 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181622+0000", 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.2185000000000002e-05 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485414+0000", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.303791372 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485547+0000", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00013326899999999999 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395356+0000", 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90980936400000001 2024-09-15T21:01:48.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:118 3.bs0 3:d87a9ad1:::smithi02732947-40:head [append 0~671744,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 84.459176925999998, 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68959868700000004, 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.448 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 118 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.449 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978444+0000", 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.978446+0000", 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.474e-06 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979140+0000", 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00069447699999999998 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979160+0000", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.9426999999999999e-05 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979174+0000", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4318000000000001e-05 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179193+0000", 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20001882100000001 2024-09-15T21:01:48.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.179380+0000", 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00018685699999999999 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.181618+0000", 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022381889999999998 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.452 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.183630+0000", 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002011905 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.667822+0000", 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.48419250200000002 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.668043+0000", 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00022071699999999999 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:120 3.bs0 3:d87a9ad1:::smithi02732947-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 84.458359087999995, 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79000348200000003, 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 120 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.454 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.455 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.455 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:48.455 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979262+0000", 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979263+0000", 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7199999999999997e-07 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979270+0000", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8369999999999999e-06 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979283+0000", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.2858e-05 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:23.979294+0000", 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0900000000000001e-05 2024-09-15T21:01:48.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278498+0000", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.29920416500000002 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.278659+0000", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00016131300000000001 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669598+0000", 2024-09-15T21:01:48.458 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.39093899100000001 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.669726+0000", 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00012752 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.769266+0000", 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.099540026000000004 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:136 3.2s0 3:4419d09e:::smithi02732947-46:head [append 0~770048,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.953098263000001, 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70671877400000005, 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 136 2024-09-15T21:01:48.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484523+0000", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.484527+0000", 2024-09-15T21:01:48.461 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8430000000000003e-06 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485847+0000", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0013201650000000001 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485887+0000", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.9901999999999998e-05 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485910+0000", 2024-09-15T21:01:48.462 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2824999999999999e-05 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486295+0000", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00038481000000000002 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486494+0000", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00019959200000000001 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.488456+0000", 2024-09-15T21:01:48.463 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0019618040000000002 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491301+0000", 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0028448119999999999 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191158+0000", 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69985731900000003 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.464 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.191242+0000", 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.3702000000000006e-05 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:137 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.951661313000002, 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80557232099999998, 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.465 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 137 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485960+0000", 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485962+0000", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3130000000000001e-06 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485973+0000", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0175e-05 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.485996+0000", 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.3320999999999999e-05 2024-09-15T21:01:48.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486013+0000", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7348000000000001e-05 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490025+0000", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00401185 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490155+0000", 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129393 2024-09-15T21:01:48.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.490819+0000", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00066498700000000002 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491857+0000", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001037418 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291463+0000", 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.79960652799999998 2024-09-15T21:01:48.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291532+0000", 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8987999999999997e-05 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:138 3.2s0 3:4419d09e:::smithi02732947-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.951583167999999, 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.113031946, 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 138 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.471 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486038+0000", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486040+0000", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.533e-06 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486047+0000", 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.8059999999999997e-06 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.472 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486067+0000", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0258000000000002e-05 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.486082+0000", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5591999999999999e-05 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491176+0000", 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0050937960000000003 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.491292+0000", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000115618 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.291883+0000", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80059077899999997 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.292024+0000", 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014110899999999999 2024-09-15T21:01:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.599070+0000", 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.307046455 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 33/28 ECSubReadReply(tid=35, attrs_read=0))", 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.849068035000002, 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90822641400000004, 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588553+0000", 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.476 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588556+0000", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.407e-06 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588581+0000", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4749999999999999e-05 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588622+0000", 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.1319000000000002e-05 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.588640+0000", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.7999000000000002e-05 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496721+0000", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.90808113199999996 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.496780+0000", 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8807000000000001e-05 2024-09-15T21:01:48.478 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:146 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~442368,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.754633217999995, 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71003215600000003, 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 146 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682988+0000", 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.480 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.682991+0000", 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.627e-06 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683279+0000", 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00028858600000000002 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683311+0000", 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1436000000000002e-05 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.481 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.683326+0000", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.5645999999999999e-05 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771339+0000", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.088012266000000006 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.771754+0000", 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00041549600000000001 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.482 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.773984+0000", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0022294670000000002 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.775928+0000", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.001944033 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.392918+0000", 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.61699003600000002 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.483 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.393020+0000", 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000102563 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:147 3.bs0 3:d37a3ed4:::smithi02732947-49:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.748223263, 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70512040600000003, 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 147 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.485 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689398+0000", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689401+0000", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.4830000000000002e-06 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689673+0000", 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00027249799999999998 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.486 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.689732+0000", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8873999999999999e-05 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789836+0000", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10010448700000001 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789992+0000", 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000155117 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.487 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.790288+0000", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00029607199999999998 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.792395+0000", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.002107046 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795527+0000", 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0031319859999999998 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394467+0000", 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59894059899999996 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.394519+0000", 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.1243999999999999e-05 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:149 3.bs0 3:d37a3ed4:::smithi02732947-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:48.489 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 83.647670289999994, 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70803958600000005, 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 149 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789951+0000", 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789952+0000", 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.0999999999999997e-07 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.491 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789957+0000", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.9760000000000003e-06 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789968+0000", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.1469e-05 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.789977+0000", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.7020000000000007e-06 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.492 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795386+0000", 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0054087170000000004 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:24.795516+0000", 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000129784 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395184+0000", 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.59966812000000003 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.493 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.395266+0000", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2509000000000006e-05 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:25.497991+0000", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10272439899999999 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:357 3.bs0 3:d1127d8d:::smithi02732947-18:head [read 0~1409024,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:48.494 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 68.237046700999997, 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.77057635199999996, 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "started", 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 357 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200575+0000", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200576+0000", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.147e-06 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200583+0000", 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1729999999999998e-06 2024-09-15T21:01:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200604+0000", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.0891e-05 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200614+0000", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0514e-05 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200811+0000", 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000196959 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.200820+0000", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.8829999999999995e-06 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205510+0000", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0046902139999999998 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.205620+0000", 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000109993 2024-09-15T21:01:48.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:00:40.971151+0000", 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.76553057800000002 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.369076192000001, 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.70767751899999998, 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068545+0000", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.068546+0000", 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.9400000000000004e-07 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069555+0000", 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0010088269999999999 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.069566+0000", 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.0834e-05 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.501 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.169651+0000", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.10008542099999999 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776159+0000", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.60650789000000005 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.776223+0000", 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.3752999999999998e-05 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpReadReply(3.2s0 69/68 ECSubReadReply(tid=189, attrs_read=0))", 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 47.272415795000001, 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71199342899999996, 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:48.503 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165206+0000", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.165208+0000", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.88e-06 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166131+0000", 2024-09-15T21:01:48.504 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00092355499999999995 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166158+0000", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.7073e-05 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.166172+0000", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4249e-05 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877145+0000", 2024-09-15T21:01:48.505 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.71097216299999999 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:01.877199+0000", 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.4509000000000003e-05 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:671 3.2s0 3:48a96439:::smithi02732947-3:head [copy-from ver 128] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 37.533676818000004, 2024-09-15T21:01:48.506 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.82993921299999995, 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 671 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903945+0000", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903948+0000", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.1370000000000002e-06 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903963+0000", 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4987000000000001e-05 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.903998+0000", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.4925000000000002e-05 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:10.904016+0000", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.8093000000000001e-05 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041691+0000", 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.13767560200000001 2024-09-15T21:01:48.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.041959+0000", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00026741900000000002 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.526650+0000", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.484691182 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.530502+0000", 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.0038516790000000002 2024-09-15T21:01:48.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733738+0000", 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20323656000000001 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:11.733884+0000", 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00014562900000000001 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:830 3.2s0 3:447dff51:::smithi02732947-14:head [copy-from ver 48] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 27.522549185999999, 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.92164203099999997, 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 830 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.512 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915072+0000", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915073+0000", 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.9300000000000006e-07 2024-09-15T21:01:48.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915079+0000", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.7559999999999996e-06 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915093+0000", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.366e-05 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:20.915101+0000", 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.6959999999999994e-06 2024-09-15T21:01:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316410+0000", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.40130822100000002 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.316742+0000", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00033288600000000001 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.525877+0000", 2024-09-15T21:01:48.515 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20913427800000001 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.634058+0000", 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.108180927 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836242+0000", 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.20218430800000001 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.516 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:21.836714+0000", 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00047230599999999997 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "osd_op(client.4641.0:963 3.2s0 3:4419d09e:::smithi02732947-46:head [setxattr _header (57) in=7b] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.767710053, 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.81706183700000001, 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.517 INFO:teuthology.orchestra.run.smithi027.stdout: "client_info": { 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "client": "client.4641", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tid": 963 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669911+0000", 2024-09-15T21:01:48.518 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669943+0000", 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.2109e-05 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669947+0000", 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669955+0000", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.1890000000000001e-06 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.669961+0000", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.0739999999999998e-06 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.670030+0000", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 6.9876999999999995e-05 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:48.520 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.670038+0000", 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.4229999999999999e-06 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.775795+0000", 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.105757378 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "started", 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.775854+0000", 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 5.8901999999999999e-05 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.521 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.776321+0000", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046652899999999998 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.883111+0000", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.106790338 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "commit_sent", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.486880+0000", 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.603769267 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.522 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.486973+0000", 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.2867000000000004e-05 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=333, last_complete=104'93, committed=1, applied=1))", 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.652135572000001, 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.802818951, 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.523 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785486+0000", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785487+0000", 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.6530000000000001e-06 2024-09-15T21:01:48.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785498+0000", 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.097e-05 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785524+0000", 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.5786e-05 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.525 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.785537+0000", 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.306e-05 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588290+0000", 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.80275311699999996 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588305+0000", 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.4365000000000001e-05 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=332, last_complete=104'91, committed=1, applied=1))", 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.539352963000001, 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68997068900000003, 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898268+0000", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898269+0000", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898299+0000", 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.9618999999999999e-05 2024-09-15T21:01:48.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898312+0000", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.3899e-05 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898321+0000", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 8.2970000000000004e-06 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588136+0000", 2024-09-15T21:01:48.529 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.68981495400000004 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588239+0000", 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000103473 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWriteReply(3.2s0 104/96 ECSubWriteReply(tid=333, last_complete=104'93, committed=1, applied=1))", 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.539287904, 2024-09-15T21:01:48.530 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.69122241100000004, 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898334+0000", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 4.5600000000000001e-07 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898337+0000", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.954e-06 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898346+0000", 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.1589999999999999e-06 2024-09-15T21:01:48.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.898353+0000", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.2130000000000002e-06 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.589092+0000", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.690738572 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.589556+0000", 2024-09-15T21:01:48.533 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.00046405699999999998 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "description": "MOSDECSubOpWrite(3.12s2 104/99 ECSubWrite(tid=175, reqid=client.4641.0:938, at_version=104'87, trim_to=0'0, roll_forward_to=103'83))", 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "initiated_at": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 10.502559196, 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.75508700799999995, 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "continuous": false, 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type_data": { 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flag_point": "reached pg", 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "events": [ 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "initiated", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "header_read", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935062+0000", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "throttled", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935064+0000", 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 1.855e-06 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "all_read", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935162+0000", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 9.7993999999999997e-05 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "dispatched", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935192+0000", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 3.0210999999999999e-05 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:37.935215+0000", 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 2.2934000000000001e-05 2024-09-15T21:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "reached_pg", 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588730+0000", 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.65351474300000001 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.588737+0000", 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 7.255e-06 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.690043+0000", 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.101305548 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: "event": "done", 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: "time": "2024-09-15T21:01:38.690149+0000", 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: "duration": 0.000106468 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-15T21:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:01:48.540 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:01:48.676 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:48.677 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:48.677 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:48.677 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:48.691 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:01:48.817 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:48.817 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:48.817 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:48.817 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:48.817 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:48.832 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:01:48.845 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.960 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:103 3.3s0 3:cdcabce2:::smithi02732947-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.186413406, 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.80825143899999996, 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 103 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.961 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770018+0000", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.770021+0000", 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6900000000000001e-06 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771045+0000", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010247909999999999 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771076+0000", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0973000000000003e-05 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771091+0000", 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4979e-05 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961435+0000", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19034404099999999 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.961594+0000", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015829700000000001 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.962906+0000", 2024-09-15T21:01:48.964 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013117840000000001 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.065386+0000", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102480339 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578156+0000", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51277011500000003 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.578269+0000", 2024-09-15T21:01:48.965 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011343 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:104 3.3s0 3:cdcabce2:::smithi02732947-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.185312039999999, 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89606025700000003, 2024-09-15T21:01:48.966 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 104 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.967 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771119+0000", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771121+0000", 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.277e-06 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771126+0000", 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7400000000000001e-06 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771137+0000", 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1056999999999999e-05 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771147+0000", 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.970 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369905+0000", 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.59875830200000002 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.370121+0000", 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021593099999999999 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.971 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.371390+0000", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012691250000000001 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377408+0000", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0060179530000000004 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667038+0000", 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.28963065599999999 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667180+0000", 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014110300000000001 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:105 3.3s0 3:cdcabce2:::smithi02732947-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.185271536000002, 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1106646549999999, 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 105 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.974 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771160+0000", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771161+0000", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5600000000000004e-07 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771165+0000", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.014e-06 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:48.975 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771178+0000", 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3356000000000001e-05 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.771186+0000", 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0639999999999994e-06 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372010+0000", 2024-09-15T21:01:48.976 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.600823569 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372132+0000", 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000121913 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667285+0000", 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29515310700000003 2024-09-15T21:01:48.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667404+0000", 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011907499999999999 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881824+0000", 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21442070099999999 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.978 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:109 3.3s0 3:cae12ce2:::smithi02732947-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.164720027000001, 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88662309900000003, 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 109 2024-09-15T21:01:48.979 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791711+0000", 2024-09-15T21:01:48.980 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.791714+0000", 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.756e-06 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792835+0000", 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0011210160000000001 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:48.981 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.792871+0000", 2024-09-15T21:01:48.982 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:01:48.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.982 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.982 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.982 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.892991+0000", 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100119631 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372191+0000", 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.479200446 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.372371+0000", 2024-09-15T21:01:48.983 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017985599999999999 2024-09-15T21:01:48.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.984 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.984 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.374107+0000", 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017366009999999999 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.985 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377410+0000", 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0033026150000000001 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678212+0000", 2024-09-15T21:01:48.986 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30080210099999999 2024-09-15T21:01:48.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.987 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.987 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:48.987 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.678334+0000", 2024-09-15T21:01:48.987 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012227000000000001 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:110 3.3s0 3:cae12ce2:::smithi02732947-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.063413170999993, 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.98706172000000003, 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 110 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.989 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893018+0000", 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893019+0000", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1510000000000001e-06 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893027+0000", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.9359999999999999e-06 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893040+0000", 2024-09-15T21:01:48.991 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2823000000000001e-05 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893049+0000", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2420000000000006e-06 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375070+0000", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48202018899999999 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.375327+0000", 2024-09-15T21:01:48.992 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025753199999999999 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.376413+0000", 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001086165 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.377756+0000", 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013425550000000001 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.993 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.879938+0000", 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50218176999999997 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880080+0000", 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014235699999999999 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:48.994 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:111 3.3s0 3:cae12ce2:::smithi02732947-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.063372780999998, 2024-09-15T21:01:48.995 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.080973239, 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:48.996 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 111 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:48.997 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:48.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893059+0000", 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6499999999999999e-07 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:48.999 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893061+0000", 2024-09-15T21:01:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1749999999999999e-06 2024-09-15T21:01:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.000 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893069+0000", 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5279999999999998e-06 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.001 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.893073+0000", 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4129999999999999e-06 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.002 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378329+0000", 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.48525598800000003 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.378462+0000", 2024-09-15T21:01:49.003 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013331899999999999 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880534+0000", 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50207104999999996 2024-09-15T21:01:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.880617+0000", 2024-09-15T21:01:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3738000000000004e-05 2024-09-15T21:01:49.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.974032+0000", 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.093414563000000006 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 32/28 ECSubWrite(tid=26, reqid=client.4641.0:92, at_version=0'0, trim_to=0'0, roll_forward_to=32'4))", 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.981275744000001, 2024-09-15T21:01:49.007 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39395627799999999, 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.008 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:49.009 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975156+0000", 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-07 2024-09-15T21:01:49.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975161+0000", 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4580000000000001e-06 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975166+0000", 2024-09-15T21:01:49.011 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.5929999999999999e-06 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975169+0000", 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5240000000000001e-06 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267529+0000", 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29235936200000001 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.012 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.267554+0000", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5675999999999999e-05 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369079+0000", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101524234 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.369112+0000", 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3200000000000001e-05 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.3s0 32/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.980466003999993, 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39257407999999999, 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:49.014 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975965+0000", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975966+0000", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.1699999999999998e-07 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975974+0000", 2024-09-15T21:01:49.015 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8150000000000006e-06 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975983+0000", 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6339999999999996e-06 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:23.975988+0000", 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6929999999999998e-06 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.016 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368525+0000", 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39253640200000001 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.368539+0000", 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/28 ECSubWrite(tid=38, reqid=client.4641.0:119, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-15T21:01:49.017 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.288757680000003, 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31587311699999998, 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.018 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667674+0000", 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667675+0000", 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7260000000000001e-06 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667702+0000", 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6227000000000001e-05 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.019 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.667728+0000", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6616999999999999e-05 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.767846+0000", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100118097 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881866+0000", 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.11401974600000001 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.881873+0000", 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8020000000000003e-06 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983508+0000", 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10163541199999999 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.983547+0000", 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8489999999999999e-05 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWriteReply(3.3s0 52/28 ECSubWriteReply(tid=27, last_complete=52'24, committed=1, applied=1))", 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 64.596561210999994, 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31660625599999997, 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359870+0000", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.023 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359872+0000", 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359885+0000", 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2585999999999999e-05 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359923+0000", 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.765e-05 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.359945+0000", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2166000000000001e-05 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676328+0000", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31638271200000001 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:44.676476+0000", 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000148832 2024-09-15T21:01:49.025 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.10.18:4 3.3s0 3:cae12ce2:::smithi02732947-37:head [assert-version v141,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 27.191944756000002, 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.32170995400000002, 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.026 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764487+0000", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764489+0000", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.593e-06 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.027 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764500+0000", 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0463e-05 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764521+0000", 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.162e-05 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764537+0000", 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5603e-05 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764721+0000", 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018443800000000001 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.764741+0000", 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9346000000000001e-05 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.029 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868783+0000", 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10404185000000001 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:21.868926+0000", 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014331200000000001 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:22.086197+0000", 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.217270729 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.030 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGRecoveryDeleteReply(3.14s0 e102,99 3:2b54c2df:::smithi02732947-23:12,71'131)", 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.885486317, 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31112111399999998, 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.031 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070945+0000", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070946+0000", 2024-09-15T21:01:49.032 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1699999999999998e-07 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070949+0000", 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.873e-06 2024-09-15T21:01:49.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.070962+0000", 2024-09-15T21:01:49.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2306999999999999e-05 2024-09-15T21:01:49.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.171073+0000", 2024-09-15T21:01:49.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100110852 2024-09-15T21:01:49.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.374553+0000", 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20348087400000001 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.382066+0000", 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0075126910000000002 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:head)])", 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.679208383000001, 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29599977599999999, 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277223+0000", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277224+0000", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4300000000000001e-06 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277233+0000", 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7760000000000003e-06 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.277255+0000", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1759999999999998e-05 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377353+0000", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100097648 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.572386+0000", 2024-09-15T21:01:49.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.19503363400000001 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.573223+0000", 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00083652800000000001 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.579673768999999, 2024-09-15T21:01:49.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40437275900000003, 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376758+0000", 2024-09-15T21:01:49.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376759+0000", 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.24e-06 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376768+0000", 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9530000000000005e-06 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376796+0000", 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8626e-05 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.376830+0000", 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3973000000000002e-05 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781072+0000", 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40424176099999998 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781130+0000", 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.8205999999999998e-05 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.579023100000001, 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.407905868, 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377408+0000", 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377410+0000", 2024-09-15T21:01:49.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.367e-06 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377422+0000", 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2376e-05 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377473+0000", 2024-09-15T21:01:49.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1422999999999999e-05 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377506+0000", 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2672999999999997e-05 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781907+0000", 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40440068499999998 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.785314+0000", 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034073440000000001 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPushReply(3.11s1 102/99 [PushReplyOp(3:8d17a01b:::smithi02732947-8:21)])", 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.578491049, 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40330712499999999, 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377940+0000", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377942+0000", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377952+0000", 2024-09-15T21:01:49.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9860000000000006e-06 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.377979+0000", 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.65e-05 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.378020+0000", 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1239999999999998e-05 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781172+0000", 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.403152332 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.781247+0000", 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5136999999999996e-05 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.bs2 104/59 ECSubWrite(tid=334, reqid=osd.3.0:334, at_version=104'102, trim_to=0'0, roll_forward_to=103'100))", 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.000894999, 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.303416089, 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955536+0000", 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955538+0000", 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.22e-06 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955577+0000", 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9696000000000001e-05 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955601+0000", 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3913e-05 2024-09-15T21:01:49.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.955616+0000", 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.446e-05 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.055847+0000", 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100231608 2024-09-15T21:01:49.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.055867+0000", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9389999999999999e-05 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.258904+0000", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20303726999999999 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.258952+0000", 2024-09-15T21:01:49.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8532000000000002e-05 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:957 3.14s0 3:2f703215:::smithi02732947-30:head [read 0~2588672,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.000338838999999, 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33119290299999998, 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.061 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 957 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956093+0000", 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956094+0000", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1459999999999999e-06 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956099+0000", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.57e-06 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.956115+0000", 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5277e-05 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.056204+0000", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10008990700000001 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.056595+0000", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00039031200000000003 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.057962+0000", 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001367098 2024-09-15T21:01:49.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.287285+0000", 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.22932359299999999 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:1014 3.14s0 3:2f703215:::smithi02732947-30:head [read 0~2588672,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:43.042431+0000", 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 5.9140001350000002, 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.33418569100000001, 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1014 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042431+0000", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042431+0000", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042432+0000", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.0399999999999996e-07 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042435+0000", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4429999999999999e-06 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042442+0000", 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7009999999999999e-06 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.042447+0000", 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4399999999999996e-06 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.044108+0000", 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016611670000000001 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.044989+0000", 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088046999999999995 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.376617+0000", 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.331627966 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDPGPush(3.19s2 112/99 [PushOp(3:9b48fc7f:::smithi02732947-22:1f, version: 97'78, data_included: [524288~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:9b48fc7f:::smithi02732947-22:1f@97'78, size: 3735552, copy_subset: [], clone_subset: {}, snapset: 23=[]:{1f: [1f, 1e, 1b, 1a, 18, 16], 23: [23, 22, 21]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-15T21:01:49.069 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:47.020503+0000", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 1.9359287620000001, 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30210995800000001, 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.020503+0000", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.020503+0000", 2024-09-15T21:01:49.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.020505+0000", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1670000000000002e-06 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.021046+0000", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054161400000000005 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.021122+0000", 2024-09-15T21:01:49.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5976000000000006e-05 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.121251+0000", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012869000000001 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.321668+0000", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.200416502 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:47.322613+0000", 2024-09-15T21:01:49.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00094500900000000002 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.073 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.074 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:01:49.208 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:49.208 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:49.208 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:49.209 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:01:49.353 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:49.353 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:49.353 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:49.353 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:49.354 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:123 3.5s0 3:ac8a7abe:::smithi02732947-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.336844378999999, 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3355707159999999, 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.504 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 123 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.505 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162756+0000", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162757+0000", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162762+0000", 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1160000000000001e-06 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.506 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162777+0000", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5119999999999999e-05 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162785+0000", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7940000000000008e-06 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474752+0000", 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31196692300000001 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.507 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.474977+0000", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000224346 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.088993+0000", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.61401634299999996 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.089139+0000", 2024-09-15T21:01:49.508 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014632600000000001 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.498327+0000", 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409187778 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:124 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~704512,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.336802012999996, 2024-09-15T21:01:49.509 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1292089329999999, 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 124 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.510 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162799+0000", 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.162800+0000", 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2999999999999999e-07 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.511 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163152+0000", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000352531 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163168+0000", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5495999999999999e-05 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163176+0000", 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6000000000000007e-06 2024-09-15T21:01:49.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475085+0000", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31190898500000003 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.475549+0000", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00046386700000000002 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.478179+0000", 2024-09-15T21:01:49.513 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0026295419999999999 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587333+0000", 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.109154746 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.291773+0000", 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70443986999999997 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.514 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292008+0000", 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023446599999999999 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:125 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.336411085999998, 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.129852265, 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.515 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 125 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:49.516 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163190+0000", 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3e-07 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163196+0000", 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7440000000000004e-06 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.517 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163208+0000", 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2143e-05 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163219+0000", 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0536999999999999e-05 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479306+0000", 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31608729600000002 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.518 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.479474+0000", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000167892 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.480552+0000", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010781899999999999 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587409+0000", 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106856267 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.519 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.292870+0000", 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70546136100000001 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293042+0000", 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017210500000000001 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.520 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:126 3.5s0 3:a50cee1c:::smithi02732947-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.336371808999999, 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4360139080000001, 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 126 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.521 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163229+0000", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163230+0000", 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8299999999999996e-07 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.522 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163234+0000", 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2969999999999997e-06 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163246+0000", 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1600000000000001e-05 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.523 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.163255+0000", 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0769999999999997e-06 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481086+0000", 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.31783108100000002 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481241+0000", 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015545199999999999 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.524 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293151+0000", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81191010100000005 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.293291+0000", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000139569 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.599243+0000", 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30595194799999997 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.525 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:130 3.5s0 3:afe51e05:::smithi02732947-44:head [append 0~737280,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.297140053999996, 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091885201, 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 130 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.526 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202461+0000", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.202463+0000", 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.131e-06 2024-09-15T21:01:49.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203138+0000", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00067539900000000003 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203157+0000", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8207e-05 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203165+0000", 2024-09-15T21:01:49.528 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6430000000000001e-06 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481296+0000", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.27813077200000003 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.481496+0000", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000199638 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484095+0000", 2024-09-15T21:01:49.529 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002599779 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587414+0000", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103319122 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294139+0000", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70672465500000003 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294346+0000", 2024-09-15T21:01:49.530 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020685500000000001 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:131 3.5s0 3:afe51e05:::smithi02732947-44:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.296417701999999, 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.091670396, 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.531 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 131 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203183+0000", 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203184+0000", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5300000000000004e-07 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203187+0000", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.931e-06 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203191+0000", 2024-09-15T21:01:49.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.442e-06 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203195+0000", 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5489999999999998e-06 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585249+0000", 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38205417400000002 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.585564+0000", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031520499999999999 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586539+0000", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00097513099999999996 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587492+0000", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0009525 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294642+0000", 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70715049900000004 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.294854+0000", 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021131200000000001 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:132 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.296401211000003, 2024-09-15T21:01:49.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.597802333, 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 132 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203200+0000", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203202+0000", 2024-09-15T21:01:49.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.64e-06 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203206+0000", 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.2080000000000002e-06 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.203210+0000", 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.586972+0000", 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.38376212100000001 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.587106+0000", 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013408199999999999 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295715+0000", 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70860878900000002 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.295946+0000", 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023066199999999999 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.801002+0000", 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50505628899999999 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:150 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.541 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.803213850999995, 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.721122539, 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 150 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696387+0000", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.696389+0000", 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.968e-06 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697166+0000", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00077724100000000002 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697201+0000", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.4989e-05 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.697217+0000", 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5764e-05 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.902636+0000", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.205419274 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.903043+0000", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040640700000000001 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.906173+0000", 2024-09-15T21:01:49.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031300350000000002 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009610+0000", 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103437511 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417378+0000", 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40776825500000002 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417510+0000", 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013109499999999999 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:151 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.796772387000004, 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715161962, 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 151 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702828+0000", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.702830+0000", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.426e-06 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703210+0000", 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00038023400000000002 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703229+0000", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8797e-05 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703238+0000", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1849999999999993e-06 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907674+0000", 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.204435696 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.907908+0000", 2024-09-15T21:01:49.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023455200000000001 2024-09-15T21:01:49.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:25.910110+0000", 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0022014449999999998 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.009614+0000", 2024-09-15T21:01:49.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099504246000000005 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417882+0000", 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40826812200000001 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.417990+0000", 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00010825899999999999 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:152 3.5s0 3:a8fd5180:::smithi02732947-50:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 84.796345985000002, 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.715476215, 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 152 2024-09-15T21:01:49.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703255+0000", 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703256+0000", 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.8899999999999999e-07 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703261+0000", 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.7930000000000003e-06 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703268+0000", 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7859999999999999e-06 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.703278+0000", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9499999999999996e-06 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007757+0000", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3044789999999999 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.007947+0000", 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00018944200000000001 2024-09-15T21:01:49.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.008984+0000", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010375930000000001 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.416568+0000", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40758359300000002 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418605+0000", 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0020369670000000002 2024-09-15T21:01:49.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:26.418731+0000", 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012640199999999999 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:352 3.5s0 3:afe51e05:::smithi02732947-44:head [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 69.357038747000004, 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.033386108, 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 352 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142562+0000", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142563+0000", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1599999999999999e-06 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142572+0000", 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4999999999999999e-06 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142593+0000", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1617e-05 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.142609+0000", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5749000000000001e-05 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243086+0000", 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10047679299999999 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.562 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.243100+0000", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3883000000000001e-05 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461803+0000", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21870289500000001 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.461929+0000", 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012663900000000001 2024-09-15T21:01:49.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:41.175948+0000", 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.71401887200000003 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:656 3.5s0 3:a8fd5180:::smithi02732947-50:head [append 0~786432] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.641371806999999, 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.88797498799999997, 2024-09-15T21:01:49.564 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 656 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858229+0000", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.858231+0000", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.097e-06 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859347+0000", 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001116105 2024-09-15T21:01:49.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859370+0000", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2289999999999998e-05 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.859380+0000", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0771000000000001e-05 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.015842+0000", 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.15646217000000001 2024-09-15T21:01:49.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.016121+0000", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027814300000000001 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.017565+0000", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001443937 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.124892+0000", 2024-09-15T21:01:49.568 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.107327906 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.745992+0000", 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.62109974199999995 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.746204+0000", 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00021182699999999999 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:661 3.5s0 3:a8fd5180:::smithi02732947-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 38.525756938999997, 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.979178514, 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.570 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 661 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973844+0000", 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973845+0000", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973848+0000", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.3340000000000002e-06 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973861+0000", 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.34e-05 2024-09-15T21:01:49.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.973870+0000", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.0229999999999992e-06 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535782+0000", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.56191254599999996 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.535923+0000", 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140882 2024-09-15T21:01:49.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748675+0000", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.21275177100000001 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.748830+0000", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000155682 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.953022+0000", 2024-09-15T21:01:49.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20419203999999999 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:764 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~507904 in=507904b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.868390294999998, 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.89954056699999996, 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.575 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 764 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631211+0000", 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631213+0000", 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2170000000000001e-06 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631857+0000", 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00064422299999999997 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631889+0000", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2246999999999998e-05 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.631905+0000", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5336e-05 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320301+0000", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68839636000000004 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.320584+0000", 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028351100000000002 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.322686+0000", 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0021011860000000001 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.326185+0000", 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.003499149 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.579 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530672+0000", 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20448741300000001 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.530751+0000", 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.8924999999999994e-05 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:765 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~475136 in=475136b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.865708425999998, 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0983793850000001, 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 765 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633892+0000", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.633894+0000", 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2249999999999999e-06 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634374+0000", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048002400000000001 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634405+0000", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.1189999999999998e-05 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634421+0000", 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5815999999999999e-05 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324182+0000", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.68976111799999995 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.324311+0000", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000129271 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.325308+0000", 2024-09-15T21:01:49.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099730799999999992 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.430267+0000", 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10495866700000001 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732131+0000", 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30186411099999999 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732272+0000", 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000140655 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:766 3.5s0 3:a50cee1c:::smithi02732947-42:head [append 0~98304] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.865071657999998, 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0981193570000001, 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.586 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 766 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634529+0000", 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634531+0000", 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4500000000000001e-06 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634708+0000", 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000177104 2024-09-15T21:01:49.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634721+0000", 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.359e-05 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634730+0000", 2024-09-15T21:01:49.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9020000000000005e-06 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427504+0000", 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.79277363899999997 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.427699+0000", 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00019504099999999999 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.590 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.429045+0000", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001346564 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.631182+0000", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.202136547 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732474+0000", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.101292148 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.732649+0000", 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000174372 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:767 3.5s0 3:a50cee1c:::smithi02732947-42:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 33.864851197999997, 2024-09-15T21:01:49.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.709944165, 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 767 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:49.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634750+0000", 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.4499999999999996e-07 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634759+0000", 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8650000000000007e-06 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634776+0000", 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6549e-05 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634783+0000", 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5340000000000002e-06 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935746+0000", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3009621600000001 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.935934+0000", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000188035 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:16.936723+0000", 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00078927499999999998 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.143378+0000", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20665556199999999 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344471+0000", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20109264199999999 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:17.344694+0000", 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022269799999999999 2024-09-15T21:01:49.597 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:954 3.16s0 3:6f7bd2df:::smithi02732947-21:head [append 0~49152] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.829982047, 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.411478311, 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.598 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 954 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669619+0000", 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669622+0000", 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.05e-06 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669745+0000", 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00012288199999999999 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669782+0000", 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6863000000000003e-05 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669806+0000", 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4646000000000001e-05 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.601 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670145+0000", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000338759 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670209+0000", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4267e-05 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.684006+0000", 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.013796691999999999 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.688884+0000", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0048775609999999999 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.841823+0000", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1529394239999999 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.842689+0000", 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00086572999999999999 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.848153+0000", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0054642909999999996 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.858398+0000", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.010245216 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.079481+0000", 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.221082785 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.081097+0000", 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001616145 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:955 3.16s0 3:6f7bd2df:::smithi02732947-21:head [setxattr _header (57) in=7b] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.8297369, 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4147843039999999, 2024-09-15T21:01:49.605 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 955 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669864+0000", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669866+0000", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.764e-06 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669876+0000", 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0159e-05 2024-09-15T21:01:49.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669905+0000", 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8649999999999998e-05 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669923+0000", 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8580000000000002e-05 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670481+0000", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055759699999999995 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670592+0000", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00011093499999999999 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.689642+0000", 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.019049916 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.609 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.690193+0000", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00055138400000000003 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.851054+0000", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1608608929999999 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.852392+0000", 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013381720000000001 2024-09-15T21:01:49.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.855581+0000", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0031888300000000001 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.859002+0000", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0034214639999999999 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.081944+0000", 2024-09-15T21:01:49.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.22294212999999999 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:39.084648+0000", 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0027038299999999999 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(osd.6.17:2 3.13s1 3:cdcabce2:::smithi02732947-35:head [copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.704602323, 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0519870250000001, 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:49.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.794999+0000", 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795002+0000", 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5370000000000002e-06 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:49.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795017+0000", 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4742999999999999e-05 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795047+0000", 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0695999999999999e-05 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795072+0000", 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4983000000000001e-05 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.795419+0000", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00034657999999999998 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for missing object", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.801594+0000", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0061751319999999998 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.846193+0000", 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0445987290000001 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:49.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.846986+0000", 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00079262500000000004 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:49.617 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:01:49.751 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:49.751 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:49.751 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:49.751 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.765 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:01:49.832 INFO:teuthology.orchestra.run.smithi027.stderr:noscrub is unset 2024-09-15T21:01:49.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:01:49.886 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:49.886 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:49.887 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:49.887 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:49.887 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:49.901 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:01:50.032 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.748571619000003, 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30140701800000003, 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.033 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280450+0000", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280451+0000", 2024-09-15T21:01:50.034 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5700000000000001e-07 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280473+0000", 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2269999999999999e-05 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280494+0000", 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0815999999999999e-05 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.035 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.280513+0000", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9236999999999999e-05 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581606+0000", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30109263600000002 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.581857+0000", 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025120200000000001 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.036 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.746841126999996, 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49991925500000001, 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.037 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282181+0000", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282182+0000", 2024-09-15T21:01:50.038 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8640000000000001e-06 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282209+0000", 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6662999999999999e-05 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282247+0000", 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7812000000000003e-05 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.039 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.282315+0000", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7966999999999995e-05 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782075+0000", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49976017 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782100+0000", 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4779000000000001e-05 2024-09-15T21:01:50.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.745593549000006, 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.49885928800000001, 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.041 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283428+0000", 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283430+0000", 2024-09-15T21:01:50.042 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4100000000000001e-06 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283469+0000", 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9613000000000002e-05 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.283492+0000", 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-05 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.043 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.383565+0000", 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100073308 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782267+0000", 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39870222999999999 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782287+0000", 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.9996999999999998e-05 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.044 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.731709445000007, 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484891562, 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:50.045 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297312+0000", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297314+0000", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.254e-06 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297327+0000", 2024-09-15T21:01:50.046 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3801999999999999e-05 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297344+0000", 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6331e-05 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.297351+0000", 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5499999999999997e-06 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.047 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782167+0000", 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.484815465 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782204+0000", 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7160000000000003e-05 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.as0 33/28 ECSubReadReply(tid=3, attrs_read=0))", 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:50.048 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.647827445000004, 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30074612499999998, 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.049 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381194+0000", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381195+0000", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381206+0000", 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1113e-05 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.050 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381214+0000", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.6899999999999998e-06 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.381220+0000", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.677e-06 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681451+0000", 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30023125000000001 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.051 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.681940+0000", 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00048899499999999999 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=34, reqid=client.4641.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.545380496999996, 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30032210199999998, 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483641+0000", 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.053 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483643+0000", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2139999999999999e-06 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483672+0000", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8915999999999999e-05 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483710+0000", 2024-09-15T21:01:50.054 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.7474000000000001e-05 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.483734+0000", 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4436000000000001e-05 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782260+0000", 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29852580000000001 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.055 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782271+0000", 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1083e-05 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783917+0000", 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0016461519999999999 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.783963+0000", 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6026999999999998e-05 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.056 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.2s2 33/28 ECSubRead(tid=35, to_read={3:48a8a569:::smithi02732947-39:head=0,4096,0}, subchunks={3:48a8a569:::smithi02732947-39:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.544461599000002, 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29835630099999999, 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:50.057 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484560+0000", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484562+0000", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.57e-06 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484580+0000", 2024-09-15T21:01:50.058 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8686e-05 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484620+0000", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.9897000000000001e-05 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.484649+0000", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8906000000000001e-05 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782582+0000", 2024-09-15T21:01:50.059 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29793278699999998 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782916+0000", 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000334455 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.2s2 33/28 ECSubWrite(tid=36, reqid=client.4641.0:136, at_version=33'13, trim_to=0'0, roll_forward_to=32'12))", 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 85.541938329999994, 2024-09-15T21:01:50.060 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.297319852, 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487083+0000", 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.061 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487085+0000", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5e-06 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487374+0000", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00028955499999999999 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487402+0000", 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7566999999999999e-05 2024-09-15T21:01:50.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.487420+0000", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7920000000000001e-05 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782954+0000", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29553405900000002 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.782963+0000", 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:01:50.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784316+0000", 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0013529919999999999 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:24.784403+0000", 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7359000000000002e-05 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.064 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=66, reqid=client.4641.0:223, at_version=0'0, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 80.450044515000002, 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.301737857, 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.065 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578977+0000", 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.578980+0000", 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4030000000000001e-06 2024-09-15T21:01:50.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579008+0000", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8394e-05 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579055+0000", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6536000000000003e-05 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.579079+0000", 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.499e-05 2024-09-15T21:01:50.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679314+0000", 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100234014 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.679337+0000", 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3186000000000001e-05 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.068 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880655+0000", 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201318514 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880715+0000", 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.982e-05 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/28 ECSubWrite(tid=68, reqid=client.4641.0:224, at_version=37'28, trim_to=0'0, roll_forward_to=36'27))", 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:50.069 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 80.447131834999993, 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29949081900000002, 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581890+0000", 2024-09-15T21:01:50.070 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581891+0000", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5540000000000001e-06 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581930+0000", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8763000000000001e-05 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581957+0000", 2024-09-15T21:01:50.071 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6373e-05 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.581973+0000", 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6140000000000001e-05 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880308+0000", 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.29833515900000002 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.072 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.880323+0000", 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5617e-05 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881326+0000", 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0010026029999999999 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:29.881381+0000", 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4610000000000001e-05 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpRead(3.5s3 48/28 ECSubRead(tid=135, to_read={3:afe51e05:::smithi02732947-44:head=0,184320,0}, subchunks={3:afe51e05:::smithi02732947-44:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 69.565582551999995, 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30175176100000001, 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.074 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463439+0000", 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463441+0000", 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6559999999999999e-06 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.075 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463459+0000", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7927e-05 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463506+0000", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.6962999999999997e-05 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.463534+0000", 2024-09-15T21:01:50.076 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7953000000000002e-05 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.764861+0000", 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30132698400000002 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:40.765191+0000", 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00033027799999999998 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.077 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.4s6 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.160461151, 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30345999400000001, 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.078 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868561+0000", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868562+0000", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0440000000000001e-06 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868595+0000", 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.324e-05 2024-09-15T21:01:50.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.868615+0000", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0593000000000001e-05 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968736+0000", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10012024899999999 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968899+0000", 2024-09-15T21:01:50.080 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016380999999999999 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.968916+0000", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6492000000000001e-05 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172000+0000", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.203084071 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.172021+0000", 2024-09-15T21:01:50.081 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0494999999999999e-05 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:502 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.389016185999999, 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30380249300000001, 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.082 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 502 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640006+0000", 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.083 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640008+0000", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8420000000000001e-06 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640020+0000", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2067e-05 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640045+0000", 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4406999999999999e-05 2024-09-15T21:01:50.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640057+0000", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1917e-05 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640154+0000", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.6767000000000004e-05 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640165+0000", 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.137e-05 2024-09-15T21:01:50.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841240+0000", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20107475499999999 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.841489+0000", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024974399999999999 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.943808+0000", 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.102318624 2024-09-15T21:01:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:503 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.388942585999999, 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308825238, 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.087 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 503 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640079+0000", 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.088 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640080+0000", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.4600000000000005e-07 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640085+0000", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.9250000000000003e-06 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640092+0000", 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.3019999999999997e-06 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.089 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640096+0000", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.1919999999999998e-06 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740388+0000", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100292057 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740400+0000", 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1420000000000001e-05 2024-09-15T21:01:50.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845387+0000", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.104987575 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.845666+0000", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000279158 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.948904+0000", 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103238063 2024-09-15T21:01:50.091 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:504 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 53.388894074, 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.409270668, 2024-09-15T21:01:50.092 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 504 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.093 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640128+0000", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.46e-07 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640133+0000", 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.4739999999999999e-06 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.094 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640145+0000", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2673e-05 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.640151+0000", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.2829999999999998e-06 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740469+0000", 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100317608 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.095 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.740477+0000", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2719999999999997e-06 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.846778+0000", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106300532 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:56.847035+0000", 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025722499999999998 2024-09-15T21:01:50.096 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:57.049398+0000", 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20236315499999999 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:770 3.as0 3:5c9d63ab:::smithi02732947-34:head [read 0~4423680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 34.394252573999999, 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30194747500000002, 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 770 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.098 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634769+0000", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634770+0000", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.6300000000000004e-07 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634775+0000", 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.8219999999999997e-06 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.099 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.634791+0000", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5994999999999998e-05 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.734865+0000", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10007408399999999 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735010+0000", 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014533800000000001 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.100 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.735283+0000", 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027298399999999999 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:15.936717+0000", 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20143348899999999 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:960 3.as0 3:5c9d63ab:::smithi02732947-34:head [copy-from ver 0] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:50.101 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.359331186, 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.177642012, 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 960 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.102 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669691+0000", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669694+0000", 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.8999999999999999e-06 2024-09-15T21:01:50.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669712+0000", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7416000000000001e-05 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669752+0000", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 4.0363e-05 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669771+0000", 2024-09-15T21:01:50.104 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8363999999999999e-05 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669936+0000", 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016557400000000001 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op must wait for map", 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669959+0000", 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3289999999999999e-05 2024-09-15T21:01:50.105 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.775174+0000", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.105214328 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.775493+0000", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031928600000000001 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847333+0000", 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.071839491 2024-09-15T21:01:50.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:961 3.as0 3:5c9d63ab:::smithi02732947-34:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 12.359213973999999, 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.17811054, 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.107 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 961 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669808+0000", 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.108 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669809+0000", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2130000000000001e-06 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669814+0000", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.1329999999999998e-06 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669829+0000", 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.543e-05 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.109 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.669838+0000", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.0000000000000002e-06 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670162+0000", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032351999999999999 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.670173+0000", 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1473e-05 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.110 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.778753+0000", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.1085791 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.778905+0000", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000152731 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847570+0000", 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0686646740000001 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.111 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847745+0000", 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00017489799999999999 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.847918+0000", 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000173368 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 104/96 ECSubWrite(tid=180, reqid=client.4641.0:985, at_version=104'59, trim_to=0'0, roll_forward_to=103'57))", 2024-09-15T21:01:50.112 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.35671097, 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30899451999999999, 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.113 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672311+0000", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672312+0000", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.026e-06 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672411+0000", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.9495999999999997e-05 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.114 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672437+0000", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5420000000000001e-05 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672459+0000", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1970000000000001e-05 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.778474+0000", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.106015363 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.115 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.778497+0000", 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.334e-05 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981218+0000", 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20272069600000001 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981305+0000", 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7208999999999998e-05 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.116 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.1s6 104/96 ECSubWrite(tid=181, reqid=client.4641.0:986, at_version=104'60, trim_to=0'0, roll_forward_to=103'57))", 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 11.356514119, 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.308874965, 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672508+0000", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.13e-07 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672536+0000", 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.8223999999999999e-05 2024-09-15T21:01:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672560+0000", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3866000000000002e-05 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.672579+0000", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8739000000000001e-05 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.781009+0000", 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10842980000000001 2024-09-15T21:01:50.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.781032+0000", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2716000000000001e-05 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981316+0000", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.200284781 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.981383+0000", 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.6225999999999999e-05 2024-09-15T21:01:50.120 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.121 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.121 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:50.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:01:50.252 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:50.252 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:50.252 INFO:teuthology.orchestra.run.smithi066.stdout: "num_ops": 0 2024-09-15T21:01:50.252 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:50.267 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:01:50.392 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:50.392 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [], 2024-09-15T21:01:50.392 INFO:teuthology.orchestra.run.smithi066.stdout: "complaint_time": 30, 2024-09-15T21:01:50.392 INFO:teuthology.orchestra.run.smithi066.stdout: "num_blocked_ops": 0 2024-09-15T21:01:50.392 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:50.407 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout: "size": 20, 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 600, 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout: "ops": [ 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.531 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 110.089608099, 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50842047700000004, 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 51 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.532 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437735+0000", 2024-09-15T21:01:50.533 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437736+0000", 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5600000000000001e-06 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.534 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437749+0000", 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.257e-05 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437771+0000", 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1610000000000001e-05 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.535 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437787+0000", 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6293000000000001e-05 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.843936+0000", 2024-09-15T21:01:50.536 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40614932300000001 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844170+0000", 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023395800000000001 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:50.537 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.844537+0000", 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036638000000000003 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846467+0000", 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001930083 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.538 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.945848+0000", 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.099381345999999995 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946014+0000", 2024-09-15T21:01:50.539 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016623199999999999 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.946155+0000", 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014112200000000001 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 110.089530033, 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51321243000000005, 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 52 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.542 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437813+0000", 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437814+0000", 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.043e-06 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437821+0000", 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.0899999999999999e-06 2024-09-15T21:01:50.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437833+0000", 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.1707e-05 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.545 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.437842+0000", 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.1130000000000007e-06 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845677+0000", 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40783460399999999 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.546 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.845814+0000", 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013704599999999999 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.846022+0000", 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020844400000000001 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.547 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.847012+0000", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00099011600000000009 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950874+0000", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.103861935 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.950960+0000", 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5789000000000001e-05 2024-09-15T21:01:50.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:00.951025+0000", 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5543000000000006e-05 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 106.299074328, 2024-09-15T21:01:50.549 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50656936200000002, 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 117 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228269+0000", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228270+0000", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228374+0000", 2024-09-15T21:01:50.551 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000104386 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228393+0000", 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8309999999999999e-05 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228409+0000", 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.6623e-05 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.552 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341639+0000", 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.113229576 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.341860+0000", 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022081499999999999 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.342104+0000", 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00024469100000000001 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.553 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.343811+0000", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001706621 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734690+0000", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.39087895499999997 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734772+0000", 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.2169999999999994e-05 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.734838+0000", 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.5864999999999994e-05 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4210.0:118 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 106.29890428500001, 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60653601999999995, 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4210", 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.66:0/2172932773", 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 118 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.556 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228439+0000", 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228440+0000", 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.35e-06 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.557 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228452+0000", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2286000000000001e-05 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228476+0000", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.3433000000000001e-05 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.228494+0000", 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8266000000000001e-05 2024-09-15T21:01:50.558 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430086+0000", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.201591982 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430217+0000", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013068199999999999 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for subops from 11", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.430418+0000", 2024-09-15T21:01:50.559 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00020088500000000001 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "op_commit", 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.433012+0000", 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0025947700000000001 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834824+0000", 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.401811853 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.560 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834910+0000", 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.5857000000000001e-05 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:04.834975+0000", 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.4655999999999999e-05 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.561 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:437 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~540672] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.898484963999998, 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54912561999999998, 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 437 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.562 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628858+0000", 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.563 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.628861+0000", 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.7499999999999999e-06 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629602+0000", 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00074142900000000005 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629633+0000", 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.0453000000000001e-05 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.564 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629649+0000", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5923e-05 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.629768+0000", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000119453 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.630003+0000", 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00023483799999999999 2024-09-15T21:01:50.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.632434+0000", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0024312940000000001 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.636553+0000", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041183879999999997 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177838+0000", 2024-09-15T21:01:50.566 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54128565900000003 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.177984+0000", 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00014543299999999999 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:438 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~229376] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.895916268000001, 2024-09-15T21:01:50.567 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54706032999999998, 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 438 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.568 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631427+0000", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631429+0000", 2024-09-15T21:01:50.569 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2630000000000002e-06 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631860+0000", 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000431261 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631886+0000", 2024-09-15T21:01:50.570 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.6230000000000001e-05 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631907+0000", 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633475+0000", 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0015675870000000001 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.571 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.633636+0000", 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00016115899999999999 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.635458+0000", 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0018221299999999999 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.637212+0000", 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0017535420000000001 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.572 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178395+0000", 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.54118301599999996 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.178487+0000", 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 9.2262999999999994e-05 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:439 3.ds0 3:b220672c:::smithi02732947-15:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.895389768000001, 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.85063377100000004, 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 439 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.574 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631953+0000", 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631956+0000", 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2730000000000001e-06 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.575 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631966+0000", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0047000000000001e-05 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.631987+0000", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.1253000000000001e-05 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732106+0000", 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.10011937999999999 2024-09-15T21:01:50.576 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179003+0000", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44689677 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179116+0000", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000112794 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.179656+0000", 2024-09-15T21:01:50.577 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00054039500000000005 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180543+0000", 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00088661899999999995 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482437+0000", 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30189397699999998 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.578 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.482587+0000", 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015026300000000001 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:440 3.ds0 3:b220672c:::smithi02732947-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 59.795192845999999, 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.75401734600000003, 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.579 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 440 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.580 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732150+0000", 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732153+0000", 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.717e-06 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732167+0000", 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.382e-05 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.581 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732202+0000", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.5553000000000003e-05 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:50.732225+0000", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2894000000000001e-05 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180004+0000", 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.44777894899999998 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.582 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.180092+0000", 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.7705000000000001e-05 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483149+0000", 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.30305710299999999 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.583 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.483259+0000", 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000109998 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:00:51.486168+0000", 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0029086070000000001 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:652 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:50.584 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.702010770999998, 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50904735300000004, 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 652 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:50.585 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825332+0000", 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825336+0000", 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.2660000000000002e-06 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.586 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825348+0000", 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2327000000000001e-05 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825385+0000", 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6918999999999997e-05 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825407+0000", 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.2560000000000001e-05 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.587 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825508+0000", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000100647 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825615+0000", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000106795 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.826443+0000", 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00082777899999999995 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.588 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044140749999999999 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334247+0000", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50339011700000003 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334380+0000", 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00013286799999999999 2024-09-15T21:01:50.589 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:653 3.ds0 3:b220672c:::smithi02732947-15:head [rollback 16] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.701894482, 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50927399799999995, 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.590 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 653 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825449+0000", 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.591 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825450+0000", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3799999999999999e-06 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825459+0000", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.9959999999999999e-06 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825480+0000", 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0930000000000001e-05 2024-09-15T21:01:50.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825502+0000", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.213e-05 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827235+0000", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.001732691 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.827493+0000", 2024-09-15T21:01:50.593 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000258139 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.828697+0000", 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0012043609999999999 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830857+0000", 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.002160201 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.594 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334605+0000", 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50374753100000003 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.334723+0000", 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000117639 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:654 3.ds0 3:b220672c:::smithi02732947-15:head [append 0~0] snapc 17=[16,15] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:50.595 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 39.701813653000002, 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50976337999999999, 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 654 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.596 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825529+0000", 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825531+0000", 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.3030000000000001e-06 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.597 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825536+0000", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 5.5670000000000002e-06 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825555+0000", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8680000000000001e-05 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.825570+0000", 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4715999999999999e-05 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.598 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830061+0000", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0044918199999999997 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830150+0000", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.8658000000000004e-05 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.830574+0000", 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00042412199999999998 2024-09-15T21:01:50.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:10.931412+0000", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100838144 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "commit_sent", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335070+0000", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40365782900000002 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:11.335293+0000", 2024-09-15T21:01:50.600 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00022254100000000001 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.632803143, 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50728277399999999, 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.601 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894540+0000", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.894542+0000", 2024-09-15T21:01:50.602 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.0099999999999998e-06 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895468+0000", 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00092661499999999999 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.895495+0000", 2024-09-15T21:01:50.603 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.686e-05 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.995597+0000", 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100101948 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401647+0000", 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.40604999000000003 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.604 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.401823+0000", 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.000175351 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=1))", 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.626949658000001, 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70345572000000001, 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.605 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900393+0000", 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.606 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900395+0000", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.553e-06 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900696+0000", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00030154100000000001 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.900721+0000", 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5018999999999999e-05 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000802+0000", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.100080497 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.603506+0000", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.60270436100000002 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.603849+0000", 2024-09-15T21:01:50.608 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00034274899999999999 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.529685687000001, 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81764706300000001, 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.609 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997657+0000", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.997658+0000", 2024-09-15T21:01:50.610 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.0470000000000001e-06 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998063+0000", 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00040452199999999999 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998087+0000", 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4320000000000001e-05 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.611 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:36.998105+0000", 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7938999999999999e-05 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814944+0000", 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81683928900000002 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815304+0000", 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00035994599999999999 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.612 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=139, attrs_read=0))", 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.526402256000001, 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50637170899999995, 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.613 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000941+0000", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.000942+0000", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.2780000000000001e-06 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.614 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001218+0000", 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00027612999999999997 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001230+0000", 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.201e-05 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.001237+0000", 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.7750000000000004e-06 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.615 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.503166+0000", 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.50192909600000002 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.507312+0000", 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0041464199999999996 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:50.616 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.441449709, 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72889100799999995, 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085893+0000", 2024-09-15T21:01:50.617 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.085895+0000", 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086206+0000", 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00031062599999999999 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.618 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086231+0000", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5363000000000001e-05 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086248+0000", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.719e-05 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814488+0000", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72824028500000004 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.619 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.814784+0000", 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00029581699999999998 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpWrite(3.5s4 103/96 ECSubWrite(tid=394, reqid=osd.5.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=103'223))", 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.441067426, 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.93721581600000003, 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.620 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086276+0000", 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.621 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086277+0000", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.443e-06 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086302+0000", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.4885999999999999e-05 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086321+0000", 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8870000000000001e-05 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.622 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.086340+0000", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8836000000000001e-05 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815358+0000", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.72901870899999999 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_started", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.815372+0000", 2024-09-15T21:01:50.623 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.4073999999999999e-05 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "sub_op_committed", 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.023407+0000", 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.208034999 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:38.023491+0000", 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 8.3999000000000007e-05 2024-09-15T21:01:50.624 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.431751319, 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.82567823699999998, 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.625 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095592+0000", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095593+0000", 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.133e-06 2024-09-15T21:01:50.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095748+0000", 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00015470100000000001 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095763+0000", 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5299999999999999e-05 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.627 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.095770+0000", 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 7.5970000000000003e-06 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.914691+0000", 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.81892052100000001 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.921270+0000", 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0065789849999999999 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.628 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "MOSDECSubOpReadReply(3.15s2 103/99 ECSubReadReply(tid=140, attrs_read=0))", 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 13.430415721999999, 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70924067599999996, 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.629 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "reached pg", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096927+0000", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.630 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.096929+0000", 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.8199999999999999e-06 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097251+0000", 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00032222400000000002 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097287+0000", 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 3.6096000000000001e-05 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.631 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.097302+0000", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.5053e-05 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.805802+0000", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.70849910699999996 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:37.806168+0000", 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00036637600000000002 2024-09-15T21:01:50.632 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "description": "osd_op(client.4641.0:1004 3.15s2 3:ad5ec547:::smithi02732947-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e108)", 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "initiated_at": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "age": 7.5931407860000002, 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.73611654900000001, 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "continuous": false, 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "type_data": { 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flag_point": "started", 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "client_info": { 2024-09-15T21:01:50.633 INFO:teuthology.orchestra.run.smithi066.stdout: "client": "client.4641", 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "tid": 1004 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "events": [ 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "initiated", 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.634 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "header_read", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934202+0000", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "throttled", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934204+0000", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.404e-06 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "all_read", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934210+0000", 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 6.0100000000000001e-06 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.635 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "dispatched", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934235+0000", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 2.5602e-05 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "queued_for_pg", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.934254+0000", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 1.836e-05 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.942267+0000", 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.0080132669999999993 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.636 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:42.942525+0000", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00025793399999999997 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "reached_pg", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.461573+0000", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.51904797800000002 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "started", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.462052+0000", 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.00047958300000000001 2024-09-15T21:01:50.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: "event": "done", 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: "time": "2024-09-15T21:01:43.670319+0000", 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: "duration": 0.20826641100000001 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-15T21:01:50.638 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-15T21:01:50.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-15T21:01:50.768 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:50.769 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:50.769 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:50.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-15T21:01:50.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T21:01:50.834 INFO:teuthology.orchestra.run.smithi027.stderr:nodeep-scrub is unset 2024-09-15T21:01:50.906 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:50.906 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:50.906 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:50.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-15T21:01:50.971 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a: {} 2024-09-15T21:01:50.971 INFO:teuthology.orchestra.run.smithi027.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:01:50.984 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b: {} 2024-09-15T21:01:50.984 INFO:teuthology.orchestra.run.smithi027.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:01:50.998 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c: {} 2024-09-15T21:01:50.998 INFO:teuthology.orchestra.run.smithi027.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:01:51.015 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T21:01:51.047 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.047 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:133 3.9s0 3:9b0039f5:::smithi02732947-45:head [append 0~753664,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.751345564000005, 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40854030299999999, 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 133 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292296+0000", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.292299+0000", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293511+0000", 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212451 2024-09-15T21:01:51.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293546+0000", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4310999999999997e-05 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293562+0000", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6371999999999998e-05 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293696+0000", 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013341500000000001 2024-09-15T21:01:51.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293944+0000", 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024839699999999997 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.296267+0000", 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002323524 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.300204+0000", 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039360649999999999 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700659+0000", 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.400455167 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.700836+0000", 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017765500000000001 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.053 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:134 3.9s0 3:9b0039f5:::smithi02732947-45:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.750042652999994, 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40777226700000002, 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.054 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 134 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293599+0000", 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293600+0000", 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.297e-06 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293606+0000", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5829999999999997e-06 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293617+0000", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293626+0000", 2024-09-15T21:01:51.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2660000000000001e-06 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.297768+0000", 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041423880000000003 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298053+0000", 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028500500000000001 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.298969+0000", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00091629999999999999 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.301049+0000", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020797659999999998 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701193+0000", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40014446799999998 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.701371+0000", 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000177775 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:135 3.9s0 3:9b0039f5:::smithi02732947-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.749962728, 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0135544839999999, 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.060 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 135 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293679+0000", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293680+0000", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9000000000000005e-07 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293685+0000", 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9389999999999996e-06 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293699+0000", 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3859000000000001e-05 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.293709+0000", 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0773e-05 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299467+0000", 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057574890000000002 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.299618+0000", 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015098999999999999 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702214+0000", 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40259595399999998 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.702480+0000", 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000266233 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307233+0000", 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60475325700000004 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:139 3.9s0 3:993457f5:::smithi02732947-47:head [append 0~786432,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.065 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.543478879000006, 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70408694900000002, 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 139 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500163+0000", 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.500164+0000", 2024-09-15T21:01:51.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7799999999999999e-06 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501497+0000", 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013322060000000001 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501523+0000", 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6673999999999999e-05 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501537+0000", 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3433e-05 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703671+0000", 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20213428899999999 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703991+0000", 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032039599999999997 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.706028+0000", 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020360619999999999 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.902496+0000", 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.196468325 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204049+0000", 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30155313700000003 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.204250+0000", 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020064700000000001 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:140 3.9s0 3:993457f5:::smithi02732947-47:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.542081433999996, 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99647449200000004, 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 140 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501560+0000", 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501561+0000", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.24e-06 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501567+0000", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3329999999999996e-06 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501581+0000", 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3925e-05 2024-09-15T21:01:51.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501588+0000", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6229999999999997e-06 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307845+0000", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80625708299999999 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308000+0000", 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015473400000000001 2024-09-15T21:01:51.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.308878+0000", 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00087798399999999995 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409994+0000", 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10111626999999999 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.497883+0000", 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087888305999999999 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498035+0000", 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000151994 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:141 3.9s0 3:993457f5:::smithi02732947-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.542042206999994, 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2013775529999999, 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 141 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501599+0000", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501600+0000", 2024-09-15T21:01:51.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.9700000000000005e-07 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501605+0000", 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6530000000000002e-06 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501618+0000", 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2656e-05 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.501626+0000", 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9829999999999996e-06 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409697+0000", 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.90807160499999995 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.409844+0000", 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000146711 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498671+0000", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.088826782000000007 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.498880+0000", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020902400000000001 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.702977+0000", 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20409724200000001 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.9s0 33/28 ECSubRead(tid=11, to_read={3:9b0039f5:::smithi02732947-45:head=0,4096,0}, subchunks={3:9b0039f5:::smithi02732947-45:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.340191739000005, 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.502076682, 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:51.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703450+0000", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703516+0000", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5759000000000005e-05 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703542+0000", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6689000000000001e-05 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205082+0000", 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50153964200000001 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205527+0000", 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000444592 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.339820923000005, 2024-09-15T21:01:51.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60343987399999999, 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703821+0000", 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703823+0000", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703850+0000", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6872e-05 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703873+0000", 2024-09-15T21:01:51.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3255e-05 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804004+0000", 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10013153900000001 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.306743+0000", 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50273875800000001 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307261+0000", 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000517414 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.339651261, 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60371858, 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.090 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703990+0000", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.703992+0000", 2024-09-15T21:01:51.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4160000000000001e-06 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704011+0000", 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9593e-05 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704032+0000", 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0428000000000002e-05 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704049+0000", 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7221999999999999e-05 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307682+0000", 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60363347599999995 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307709+0000", 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.339243070999999, 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50168002899999997, 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704399+0000", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704401+0000", 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0360000000000001e-06 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704429+0000", 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8411999999999998e-05 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704458+0000", 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8881e-05 2024-09-15T21:01:51.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.704477+0000", 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9542e-05 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205923+0000", 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50144549100000002 2024-09-15T21:01:51.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.206079+0000", 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015566700000000001 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.271963227000001, 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43402687899999998, 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771678+0000", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771680+0000", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9570000000000001e-06 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771704+0000", 2024-09-15T21:01:51.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3642000000000002e-05 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771741+0000", 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7387e-05 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.771761+0000", 2024-09-15T21:01:51.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9335999999999999e-05 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205590+0000", 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.43382923899999998 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.205705+0000", 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000115318 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpReadReply(3.9s0 33/28 ECSubReadReply(tid=11, attrs_read=0))", 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.240750324999993, 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50445262700000004, 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802891+0000", 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802893+0000", 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9099999999999999e-06 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802914+0000", 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1245000000000001e-05 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802943+0000", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8158e-05 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.802964+0000", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1138999999999999e-05 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307315+0000", 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50435145100000001 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.307344+0000", 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8724000000000001e-05 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 33/28 ECSubWriteReply(tid=12, last_complete=33'7, committed=1, applied=1))", 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.238981455000001, 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39900848, 2024-09-15T21:01:51.107 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804660+0000", 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804661+0000", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3599999999999999e-06 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804670+0000", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2029999999999992e-06 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804694+0000", 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3828e-05 2024-09-15T21:01:51.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.804713+0000", 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9902999999999999e-05 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203594+0000", 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39888079999999998 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.203669+0000", 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4386000000000002e-05 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:166 3.9s0 3:9b48fc7f:::smithi02732947-22:head [copy-from ver 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.138956973999996, 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.71698077900000001, 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 166 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904685+0000", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904687+0000", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8870000000000001e-06 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904696+0000", 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9899999999999992e-06 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904709+0000", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2543e-05 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904717+0000", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904853+0000", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013647299999999999 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.904974+0000", 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120728 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.115180+0000", 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210206536 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.116911+0000", 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001730517 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621588+0000", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.504676919 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:26.621665+0000", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7520000000000003e-05 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:466 3.9s0 3:9b48fc7f:::smithi02732947-22:head [rollback e] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:51.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.301833301000002, 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36587998700000002, 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 466 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:51.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741808+0000", 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741810+0000", 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7e-06 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741817+0000", 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2099999999999996e-06 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741838+0000", 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0659e-05 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741854+0000", 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5930000000000002e-05 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802675+0000", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.060821433000000001 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.802967+0000", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029195399999999998 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804598+0000", 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016307870000000001 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809146+0000", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.004548191 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107499+0000", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298352532 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107688+0000", 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018959099999999999 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:467 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~0] snapc f=[e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.301760963, 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36615330200000001, 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.123 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 467 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741881+0000", 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741882+0000", 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3e-06 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741888+0000", 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9170000000000003e-06 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741910+0000", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1838999999999999e-05 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.741928+0000", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8016000000000002e-05 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809749+0000", 2024-09-15T21:01:51.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.067821091 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.809897+0000", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000148312 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810785+0000", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00088809800000000003 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.812435+0000", 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0016501619999999999 2024-09-15T21:01:51.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107930+0000", 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.295494543 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.108034+0000", 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010402400000000001 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.128 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:506 3.9s0 3:9b48fc7f:::smithi02732947-22:head [append 0~540672] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.361764006000001, 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.37081094999999997, 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 506 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681878+0000", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.681880+0000", 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3300000000000001e-06 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682876+0000", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099578100000000005 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682908+0000", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2138999999999998e-05 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.682923+0000", 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5577e-05 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683044+0000", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121033 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.683056+0000", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1732999999999999e-05 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.746713+0000", 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.063656931999999999 2024-09-15T21:01:51.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.747027+0000", 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031414700000000003 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.749307+0000", 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0022794339999999999 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.841341+0000", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.092034070999999995 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052501+0000", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.21116006500000001 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.052689+0000", 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000187708 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.134 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:508 3.9s0 3:9b48fc7f:::smithi02732947-22:head [setxattr _header (57) in=7b] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.352703439999999, 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.459624861, 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 508 2024-09-15T21:01:51.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690938+0000", 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690939+0000", 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.257e-06 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690947+0000", 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6990000000000003e-06 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690958+0000", 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.128e-05 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.690967+0000", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691124+0000", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015739899999999999 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting_for_map not empty", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.691145+0000", 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0846e-05 2024-09-15T21:01:51.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942749+0000", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25160399300000003 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.942917+0000", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167522 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.943821+0000", 2024-09-15T21:01:51.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090415099999999996 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.945525+0000", 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0017036480000000001 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150355+0000", 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20482993199999999 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.150563+0000", 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020850400000000001 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:624 3.9s0 3:9b0039f5:::smithi02732947-45:head [create,setxattr [OXS[Rf]kSmSVZkcqrPoTRUsq[o]epTblNRaQggt (1858) in=40b,setxattr QrooaVmb\\N]q[ho]kgWWcr\\eocaQRtZ_`]Qsipqt (174) in=40b,setxattr Ydefbt^YQ\\dpmjectRYX^qRe[eQdQqbZhllc (770) in=36b,setxattr TV\\gUfWjiSV[PabStYoPOXcXUVsSaXdhhRnnhpfs (454) in=40b,setxattr m_O\\\\icSfcPOnoNRZ]O`X]X`ZaVOXegXNVmPglmp (734) in=40b,setxattr mf`_l`a[Y`agdjhtOpoWmnWT^dS[ScSdZWamrcjp (699) in=40b,setxattr hPm]lVO[\\[iefNWiNnaWNQr (770) in=23b,setxattr gkna_]qPX[S`[_g_ifYo]hUlgScp_jnl[^WpdheP (1614) in=40b,setxattr YSndfaQqlVQQXmZiNjPrZZa`glYedsbkkcroSttS (1333) in=40b,setxattr VVliPgssabe (770) in=11b,setxattr kdelpR\\kNn^\\r\\aeabqa\\jcQSaOXgNlVTroTaodb (1610) in=40b,setxattr a[YZZ[_nQae][Ujatlhnen_QorP_PROiQkaafPl^ (1754) in=40b,setxattr Y\\iYcXa`paqfZggTnSmaVZ[Yt`te\\rdepg^foUUX (12280) in=40b,setxattr WPicPk_V\\Z]`USR]\\Xq (770) in=19b,setxattr gqtnZjUUjhkmnQqhohYVhkliPOppPSXTshf_sVbq (877) in=40b,setxattr Nadgio[acb^P_fdps]bigQrctbcktsZdTe]Z_hhb (8628) in=40b,setxattr oUUbUt^qtWedq]`NoRbZf[jPhdNrRbjTU`m^jbpe (666) in=40b,setxattr UUhYqmbd_S[all[PkPg\\odhP\\gkshjetOj][^OWq (16140) in=40b,setxattr WSTftae_cS`q^sZiZQskrTnpRX^Tkmid^ZbYOeTo (1997) in=40b,setxattr bdcqYYhcTQq^O[TjlUb[oTXa_hoaQStVglem^VqX (981) in=40b] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.406781737999999, 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50056462599999996, 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 624 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636860+0000", 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.636862+0000", 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637027+0000", 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016484200000000001 2024-09-15T21:01:51.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637047+0000", 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0182000000000001e-05 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.637059+0000", 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1892999999999999e-05 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745449+0000", 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10838987999999999 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.745728+0000", 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027862699999999999 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.747309+0000", 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001581004 2024-09-15T21:01:51.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.748624+0000", 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013150099999999999 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137271+0000", 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.38864758999999999 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:06.137424+0000", 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015299899999999999 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:909 3.9s0 3:9b0039f5:::smithi02732947-45:head [copy-from ver 155] snapc 21=[21,1f,1e,1d,1b,1a,18,16] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.422741658, 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.57516277900000001, 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 909 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620900+0000", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620903+0000", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9299999999999999e-06 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620915+0000", 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2612999999999999e-05 2024-09-15T21:01:51.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620948+0000", 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2617999999999998e-05 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.620965+0000", 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6625000000000001e-05 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.715178+0000", 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.094213704999999995 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.715659+0000", 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000480774 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.909645+0000", 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19398601500000001 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:27.917783+0000", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0081374810000000002 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:28.195653+0000", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.27787013999999999 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:28.196063+0000", 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00040987800000000002 2024-09-15T21:01:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.153 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-15T21:01:51.280 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:51.280 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-15T21:01:51.416 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.416 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:51.416 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:51.416 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:51.416 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-15T21:01:51.554 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 39/28 ECSubWrite(tid=38, reqid=client.4641.0:247, at_version=39'27, trim_to=0'0, roll_forward_to=38'25))", 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.17793562, 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30124132199999998, 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373153+0000", 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373155+0000", 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.063e-06 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373386+0000", 2024-09-15T21:01:51.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023070100000000001 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.373418+0000", 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2378000000000001e-05 2024-09-15T21:01:51.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473583+0000", 2024-09-15T21:01:51.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100164772 2024-09-15T21:01:51.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473814+0000", 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023125000000000001 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.473832+0000", 2024-09-15T21:01:51.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7949999999999999e-05 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674322+0000", 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20048924200000001 2024-09-15T21:01:51.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:31.674395+0000", 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2966000000000006e-05 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=4, reqid=client.4641.0:429, at_version=59'74, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:51.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:51.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.787539727999999, 2024-09-15T21:01:51.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30705592100000001, 2024-09-15T21:01:51.566 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763549+0000", 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763551+0000", 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.12e-06 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763784+0000", 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023310900000000001 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763818+0000", 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3855e-05 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.763835+0000", 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6393e-05 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964212+0000", 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.200376947 2024-09-15T21:01:51.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.964244+0000", 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2035000000000001e-05 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070501+0000", 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10625775599999999 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.070605+0000", 2024-09-15T21:01:51.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103706 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=5, reqid=client.4641.0:430, at_version=59'75, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.782265399000003, 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.402708554, 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768823+0000", 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768826+0000", 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4600000000000002e-06 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769062+0000", 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023581599999999999 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769156+0000", 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4553000000000001e-05 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.769216+0000", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9932000000000001e-05 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966630+0000", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19741336600000001 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.966643+0000", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3609e-05 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171495+0000", 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20485210600000001 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171532+0000", 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6711999999999997e-05 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 59/57 ECSubWrite(tid=6, reqid=client.4641.0:431, at_version=59'76, trim_to=0'0, roll_forward_to=58'72))", 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:51.577 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.679295353999997, 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299792744, 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871794+0000", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7299999999999996e-07 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871820+0000", 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5873000000000001e-05 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871847+0000", 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6792e-05 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.871873+0000", 2024-09-15T21:01:51.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6075999999999999e-05 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068219+0000", 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19634641799999999 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.068236+0000", 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6912999999999999e-05 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171560+0000", 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10332366 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171586+0000", 2024-09-15T21:01:51.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6438999999999999e-05 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpRead(3.4s2 59/57 ECSubRead(tid=9, to_read={3:2b54c2df:::smithi02732947-23:head=0,483328,0}, subchunks={3:2b54c2df:::smithi02732947-23:head=[0,1]}, attrs_to_read=))", 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.679040209999997, 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29954117000000002, 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872049+0000", 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4700000000000002e-07 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872055+0000", 2024-09-15T21:01:51.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8370000000000002e-06 2024-09-15T21:01:51.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872067+0000", 2024-09-15T21:01:51.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2493e-05 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.872079+0000", 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1456e-05 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171223+0000", 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.299144415 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.171590+0000", 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036652199999999998 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=57, reqid=client.4641.0:465, at_version=60'37, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.750284978000003, 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30393555799999999, 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800804+0000", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0699999999999997e-07 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800860+0000", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5575000000000002e-05 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800883+0000", 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3116e-05 2024-09-15T21:01:51.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.800903+0000", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9426e-05 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001512+0000", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20060972299999999 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.001540+0000", 2024-09-15T21:01:51.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7674999999999998e-05 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104682+0000", 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.103142055 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104739+0000", 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7481000000000001e-05 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=58, reqid=client.4641.0:466, at_version=60'38, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.746421853999998, 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30238109299999999, 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804667+0000", 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804669+0000", 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7969999999999999e-06 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804721+0000", 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1820999999999997e-05 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804762+0000", 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1115999999999997e-05 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.804786+0000", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4352e-05 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103533+0000", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29874738099999998 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103546+0000", 2024-09-15T21:01:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2768e-05 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.106961+0000", 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003415094 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107048+0000", 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6763999999999998e-05 2024-09-15T21:01:51.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 60/28 ECSubWrite(tid=59, reqid=client.4641.0:467, at_version=60'39, trim_to=0'0, roll_forward_to=56'35))", 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.740268667999999, 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29628899199999997, 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810820+0000", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810822+0000", 2024-09-15T21:01:51.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0430000000000002e-06 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810872+0000", 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9962e-05 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810913+0000", 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0642000000000002e-05 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.810940+0000", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6662e-05 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104296+0000", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29335650600000002 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.104312+0000", 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5559000000000001e-05 2024-09-15T21:01:51.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107057+0000", 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027455829999999998 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107109+0000", 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2034999999999999e-05 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=15, reqid=client.4641.0:461, at_version=60'82, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.651117898999999, 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20751788099999999, 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899971+0000", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.899973+0000", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.08e-06 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900092+0000", 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000118466 2024-09-15T21:01:51.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900123+0000", 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1149000000000003e-05 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900146+0000", 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3779000000000001e-05 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900331+0000", 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000184594 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900362+0000", 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0559000000000003e-05 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107409+0000", 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20704719299999999 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107489+0000", 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0061000000000005e-05 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=16, reqid=client.4641.0:462, at_version=60'83, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.650893050000001, 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20735974700000001, 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900196+0000", 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900197+0000", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0139999999999999e-06 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900419+0000", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022236900000000001 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900447+0000", 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7373e-05 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900472+0000", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4981999999999999e-05 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002446+0000", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101974907 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.002464+0000", 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7045999999999999e-05 2024-09-15T21:01:51.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107496+0000", 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.105032797 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107556+0000", 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9259000000000003e-05 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 60/57 ECSubWrite(tid=17, reqid=client.4641.0:463, at_version=60'84, trim_to=0'0, roll_forward_to=59'76))", 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.650570747000003, 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20715383400000001, 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900518+0000", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900520+0000", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.44e-06 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900569+0000", 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9608999999999999e-05 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900593+0000", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4034999999999999e-05 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.900613+0000", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9562999999999999e-05 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103732+0000", 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.203119718 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.103749+0000", 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6759e-05 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107564+0000", 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0038144160000000002 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:52.107672+0000", 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000108294 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.616 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.9s3 73/72 ECSubWrite(tid=91, reqid=osd.8.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=73'55))", 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.175668780000002, 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29881405799999999, 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375420+0000", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375421+0000", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0899999999999999e-06 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375434+0000", 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2388000000000001e-05 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375447+0000", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2979000000000001e-05 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.375456+0000", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8870000000000004e-06 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572431+0000", 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19697500200000001 2024-09-15T21:01:51.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.572450+0000", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8260000000000001e-05 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674174+0000", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10172400700000001 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:05.674234+0000", 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0445000000000001e-05 2024-09-15T21:01:51.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=398, reqid=osd.5.0:398, at_version=104'95, trim_to=0'0, roll_forward_to=103'91))", 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.867541781, 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.303411086, 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683547+0000", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683549+0000", 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7129999999999999e-06 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683602+0000", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3097999999999999e-05 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683643+0000", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1372000000000003e-05 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.683666+0000", 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3079e-05 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.684664+0000", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00099762700000000006 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.684675+0000", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1342000000000001e-05 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.986889+0000", 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30221355700000002 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.986958+0000", 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9298e-05 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=128, reqid=osd.10.0:128, at_version=104'183, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.865049955, 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50142837399999995, 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.625 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686039+0000", 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9700000000000002e-07 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686063+0000", 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3509000000000001e-05 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.686076+0000", 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3621999999999999e-05 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.786146+0000", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100069362 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.885328+0000", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.099182114000000002 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.885344+0000", 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5911999999999999e-05 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187436+0000", 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.302092527 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187467+0000", 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0930999999999999e-05 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=400, reqid=client.4641.0:968, at_version=104'97, trim_to=0'0, roll_forward_to=103'91))", 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.757253331999999, 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39368508299999999, 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793836+0000", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793837+0000", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.034e-06 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793943+0000", 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106263 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793957+0000", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4535000000000001e-05 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.793967+0000", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.2690000000000005e-06 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985418+0000", 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19145102 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985435+0000", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7172000000000001e-05 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187484+0000", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20204925800000001 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187521+0000", 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6532000000000002e-05 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.633 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=129, reqid=client.4641.0:974, at_version=104'185, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.725708334, 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.36217170399999998, 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:51.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825381+0000", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825383+0000", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7099999999999999e-06 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825437+0000", 2024-09-15T21:01:51.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3446999999999998e-05 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825458+0000", 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1579000000000001e-05 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.825474+0000", 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5277999999999998e-05 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985924+0000", 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.16045072099999999 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.985940+0000", 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5705999999999998e-05 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187525+0000", 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20158479400000001 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187552+0000", 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7469000000000001e-05 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.4s2 104/96 ECSubWrite(tid=130, reqid=client.4641.0:975, at_version=104'186, trim_to=0'0, roll_forward_to=103'181))", 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.720944121, 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.35784861000000001, 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830145+0000", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830146+0000", 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.217e-06 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830176+0000", 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0188e-05 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830190+0000", 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4341e-05 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830198+0000", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3379999999999999e-06 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187198+0000", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.35700012599999997 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187207+0000", 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0990000000000005e-06 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187953+0000", 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074618899999999997 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187993+0000", 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0111999999999998e-05 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.cs4 104/28 ECSubWrite(tid=404, reqid=client.4641.0:969, at_version=104'98, trim_to=0'0, roll_forward_to=104'93))", 2024-09-15T21:01:51.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.646756749, 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.28325234199999999, 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904332+0000", 2024-09-15T21:01:51.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904333+0000", 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8800000000000002e-07 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904350+0000", 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6988999999999999e-05 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904362+0000", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2149e-05 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.904374+0000", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1827999999999999e-05 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.988290+0000", 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.083916404 2024-09-15T21:01:51.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.988317+0000", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6444999999999998e-05 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187556+0000", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.19923970299999999 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.187584+0000", 2024-09-15T21:01:51.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8136e-05 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs7 108/59 ECSubWrite(tid=341, reqid=client.4641.0:1010, at_version=108'105, trim_to=0'0, roll_forward_to=104'102))", 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:43.140055+0000", 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4110334620000007, 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20447367299999999, 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140055+0000", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140055+0000", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140056+0000", 2024-09-15T21:01:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0100000000000001e-06 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140102+0000", 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5652000000000003e-05 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140124+0000", 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1605999999999999e-05 2024-09-15T21:01:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140147+0000", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3132000000000001e-05 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.242680+0000", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102532927 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.242697+0000", 2024-09-15T21:01:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7166000000000001e-05 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.344465+0000", 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101768082 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.344529+0000", 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4097999999999995e-05 2024-09-15T21:01:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "MOSDECSubOpWrite(3.bs7 108/59 ECSubWrite(tid=342, reqid=client.4641.0:1011, at_version=108'106, trim_to=0'0, roll_forward_to=104'102))", 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:43.140179+0000", 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.4109095329999999, 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20586517600000001, 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140179+0000", 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140179+0000", 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140180+0000", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.054e-06 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140224+0000", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3231e-05 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140245+0000", 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1829999999999999e-05 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.140264+0000", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8216e-05 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.344091+0000", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20382737200000001 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.344108+0000", 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7122e-05 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.345980+0000", 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001871827 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:43.346045+0000", 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4523999999999994e-05 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:51.657 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-15T21:01:51.786 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:51.787 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:51.787 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-15T21:01:51.844 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T21:01:51.870 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T21:01:51.919 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:51.920 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:51.920 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:51.920 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:51.920 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:51.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-15T21:01:52.060 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:52.060 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:430 3.4s0 3:22a20f07:::smithi02732947-13:head [append 0~770048] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.446104448, 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67855575999999995, 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.061 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 430 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610257+0000", 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.610260+0000", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8870000000000001e-06 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611394+0000", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011336479999999999 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611427+0000", 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3259000000000001e-05 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611443+0000", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5549999999999999e-05 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.765717+0000", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15427427099999999 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.766028+0000", 2024-09-15T21:01:52.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031123899999999999 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.768367+0000", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023391079999999999 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858789+0000", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.090422016999999993 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288681+0000", 2024-09-15T21:01:52.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.42989205899999999 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.288813+0000", 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000131722 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:431 3.4s0 3:22a20f07:::smithi02732947-13:head [setxattr _header (57) in=7b] snapc f=[f,e,d,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:52.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.444890565000001, 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.77799016399999998, 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 431 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:52.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611471+0000", 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611472+0000", 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.176e-06 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611480+0000", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2269999999999996e-06 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611491+0000", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1776e-05 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611500+0000", 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1929999999999993e-06 2024-09-15T21:01:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770200+0000", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15870036200000001 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.770513+0000", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031272599999999999 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.771421+0000", 2024-09-15T21:01:52.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00090811600000000004 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.858791+0000", 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.087370619999999996 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389309+0000", 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.53051743900000004 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.389461+0000", 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015252899999999999 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:432 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.444848694000001, 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78326424900000002, 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.072 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 432 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611513+0000", 2024-09-15T21:01:52.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611514+0000", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.004e-06 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611517+0000", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4029999999999999e-06 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611528+0000", 2024-09-15T21:01:52.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0876000000000001e-05 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.611536+0000", 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7000000000000008e-06 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772246+0000", 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.160710251 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:50.772520+0000", 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027356700000000001 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390423+0000", 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61790300099999995 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.390689+0000", 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026639399999999998 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:51.394777+0000", 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0040880530000000003 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:576 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~786432] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.450435450000001, 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61536784200000005, 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 576 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605926+0000", 2024-09-15T21:01:52.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.605928+0000", 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1299999999999999e-06 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607006+0000", 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001077862 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607032+0000", 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5885999999999999e-05 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.607045+0000", 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3293e-05 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662488+0000", 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055442577999999999 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.662791+0000", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030328799999999997 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.665971+0000", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031791979999999998 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816856+0000", 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15088531299999999 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221142+0000", 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40428599700000001 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221294+0000", 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000152297 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:577 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~376832] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:52.082 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.447128687999999, 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61247240599999997, 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 577 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:52.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609233+0000", 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.609236+0000", 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6460000000000002e-06 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610212+0000", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00097600200000000001 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610242+0000", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9963000000000001e-05 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610268+0000", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5939e-05 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712231+0000", 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101962937 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.712411+0000", 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018002200000000001 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.714107+0000", 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696896 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.816859+0000", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102751021 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221545+0000", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40468618099999998 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221705+0000", 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016079899999999999 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:578 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 12=[12] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.446052399000003, 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61180690699999996, 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.088 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 578 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610309+0000", 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610311+0000", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9800000000000001e-06 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610321+0000", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9690000000000006e-06 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610342+0000", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0375999999999999e-05 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.610357+0000", 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5237e-05 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817100+0000", 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20674321900000001 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.817311+0000", 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021106099999999999 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.818686+0000", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013748009999999999 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:01.919851+0000", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101164695 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.221947+0000", 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30209587999999998 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:02.222116+0000", 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016968899999999999 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:774 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~573440] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.399007253999997, 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.79679720700000001, 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.093 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 774 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657355+0000", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.657358+0000", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0939999999999999e-06 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658209+0000", 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00085119500000000001 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658243+0000", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3707000000000001e-05 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.658259+0000", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6985000000000001e-05 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837273+0000", 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.17901335199999999 2024-09-15T21:01:52.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.837446+0000", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017337400000000001 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.840435+0000", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029892199999999999 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853927+0000", 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013491981 2024-09-15T21:01:52.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.453952+0000", 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60002478699999995 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.454152+0000", 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000199512 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:775 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.395606591000004, 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89396708599999997, 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 775 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660755+0000", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.660758+0000", 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5100000000000001e-06 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661604+0000", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084594499999999997 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661636+0000", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2280000000000003e-05 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.661651+0000", 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4962e-05 2024-09-15T21:01:52.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844564+0000", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18291332699999999 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.844791+0000", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000226646 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.846827+0000", 2024-09-15T21:01:52.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020363400000000002 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853934+0000", 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071071449999999996 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554542+0000", 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70060748500000003 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554722+0000", 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018044600000000001 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:776 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~622592] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.392124498000001, 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.89087423499999996, 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.104 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 776 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664237+0000", 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.664240+0000", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4930000000000001e-06 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665080+0000", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00084009500000000001 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665112+0000", 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1961000000000001e-05 2024-09-15T21:01:52.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.665127+0000", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5228999999999999e-05 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848176+0000", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.18304937199999999 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.848475+0000", 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00029845799999999998 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.850672+0000", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0021974080000000001 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853936+0000", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0032637159999999998 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.554980+0000", 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70104405299999994 2024-09-15T21:01:52.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555112+0000", 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013145 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:777 3.4s0 3:29dd4e88:::smithi02732947-38:head [append 0~311296] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.390346211000001, 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88944256600000005, 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 777 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666016+0000", 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666018+0000", 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3159999999999999e-06 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666502+0000", 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048393600000000002 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.666534+0000", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2255000000000002e-05 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766637+0000", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100103422 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851711+0000", 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.085073812999999998 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.851901+0000", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018974899999999999 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.853195+0000", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012942540000000001 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.854601+0000", 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001405445 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555319+0000", 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70071828300000005 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.555458+0000", 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013909299999999999 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:778 3.4s0 3:29dd4e88:::smithi02732947-38:head [setxattr _header (57) in=7b] snapc 19=[19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.289687585000003, 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.88938152199999998, 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 778 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766674+0000", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766676+0000", 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9400000000000001e-06 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766686+0000", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0249e-05 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.766707+0000", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0879000000000001e-05 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866809+0000", 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100102237 2024-09-15T21:01:52.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866970+0000", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160848 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867191+0000", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000220576 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.867924+0000", 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00073345699999999997 2024-09-15T21:01:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868858+0000", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093380199999999996 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.655861+0000", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78700323500000002 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.656056+0000", 2024-09-15T21:01:52.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019429899999999999 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:779 3.4s0 3:29dd4e88:::smithi02732947-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.189508386999997, 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.99582967, 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 779 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866853+0000", 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866856+0000", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.463e-06 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866868+0000", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2398999999999999e-05 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866896+0000", 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7370999999999999e-05 2024-09-15T21:01:52.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.866914+0000", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8788999999999999e-05 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868425+0000", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001510174 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:15.868575+0000", 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000150109 2024-09-15T21:01:52.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657374+0000", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.78879953300000005 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.657536+0000", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016209800000000001 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.862683+0000", 2024-09-15T21:01:52.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.205146734 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:783 3.4s0 3:235c4434:::smithi02732947-36:head [read 0~4751360,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 35.418396160999997, 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.636081864, 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.125 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 783 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637966+0000", 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637967+0000", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2899999999999999e-06 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637973+0000", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1399999999999997e-06 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.637991+0000", 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7825e-05 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:16.638004+0000", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3213e-05 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039573+0000", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.40156882999999999 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.039782+0000", 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020919100000000001 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:17.274047+0000", 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.234265375 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:838 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~737280] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:52.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.329760804999999, 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61883177, 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 838 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726601+0000", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.726605+0000", 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7730000000000001e-06 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727781+0000", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011759159999999999 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727819+0000", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8733999999999999e-05 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.727838+0000", 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8811999999999999e-05 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728035+0000", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00019682099999999999 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.728056+0000", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0971999999999998e-05 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.731965+0000", 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039089399999999996 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732120+0000", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015466000000000001 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.734867+0000", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027471489999999999 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946633+0000", 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.211766446 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345294+0000", 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39866105299999999 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345433+0000", 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013849399999999999 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:839 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~770048] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.324160805999998, 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61379308200000005, 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 839 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732201+0000", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.732204+0000", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6560000000000001e-06 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733438+0000", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001234392 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733471+0000", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3414999999999998e-05 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.733491+0000", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9698999999999999e-05 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837640+0000", 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.104148692 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.837807+0000", 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000167195 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.840395+0000", 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025880370000000001 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.946894+0000", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.106498578 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345806+0000", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39891273999999999 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.345994+0000", 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00018767799999999999 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:840 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~540672] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.320568259000002, 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61059644300000004, 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 840 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.141 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735794+0000", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.735797+0000", 2024-09-15T21:01:52.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.67e-06 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736891+0000", 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010939680000000001 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736923+0000", 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1653999999999999e-05 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.736939+0000", 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6045999999999998e-05 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943125+0000", 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20618578000000001 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.943433+0000", 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030832599999999999 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.945306+0000", 2024-09-15T21:01:52.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0018727699999999999 2024-09-15T21:01:52.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.047848+0000", 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102542726 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346245+0000", 2024-09-15T21:01:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29839692899999998 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346390+0000", 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014457399999999999 2024-09-15T21:01:52.148 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:841 3.4s0 3:261a50f9:::smithi02732947-12:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.149 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.318652232000002, 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60917734099999998, 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 841 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737710+0000", 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.737713+0000", 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0230000000000001e-06 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738201+0000", 2024-09-15T21:01:52.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048861200000000003 2024-09-15T21:01:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738252+0000", 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0773000000000003e-05 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738276+0000", 2024-09-15T21:01:52.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3825e-05 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.148832+0000", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41055591200000002 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.149003+0000", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017122200000000001 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.150384+0000", 2024-09-15T21:01:52.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013808959999999999 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154322+0000", 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039383689999999997 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346706+0000", 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.192384097 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.346887+0000", 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000180612 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:842 3.4s0 3:261a50f9:::smithi02732947-12:head [setxattr _header (57) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.318045029, 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.70825646600000003, 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 842 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738317+0000", 2024-09-15T21:01:52.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738318+0000", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7269999999999999e-06 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738328+0000", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7599999999999997e-06 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738350+0000", 2024-09-15T21:01:52.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1471000000000002e-05 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738394+0000", 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4484000000000003e-05 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151567+0000", 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.413173238 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.151818+0000", 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025079399999999998 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153019+0000", 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012006740000000001 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.154654+0000", 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00163489 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446326+0000", 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29167209999999999 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.446573+0000", 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000247328 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:843 3.4s0 3:261a50f9:::smithi02732947-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.317937405999999, 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.91369339800000005, 2024-09-15T21:01:52.163 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 843 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738424+0000", 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738426+0000", 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4169999999999999e-06 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738435+0000", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0100000000000001e-06 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738461+0000", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6605e-05 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.738477+0000", 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5364999999999999e-05 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153751+0000", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.41527431799999998 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.153990+0000", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000238698 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447565+0000", 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29357503600000001 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.447832+0000", 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026724700000000002 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.652118+0000", 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20428570200000001 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:976 3.4s0 3:22a20f07:::smithi02732947-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333922806, 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.86952956000000003, 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.169 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 976 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722439+0000", 2024-09-15T21:01:52.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722440+0000", 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0499999999999999e-06 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722444+0000", 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1030000000000004e-06 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722458+0000", 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3712e-05 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.722466+0000", 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1440000000000003e-06 2024-09-15T21:01:52.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830570+0000", 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10810450000000001 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.830715+0000", 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014405 2024-09-15T21:01:52.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.389970+0000", 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.55925579599999997 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.390085+0000", 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000114455 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.591969+0000", 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20188375 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.175 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:52.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-15T21:01:52.306 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:52.307 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2024-09-15T21:01:52.307 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:52.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-15T21:01:52.436 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:52.436 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2024-09-15T21:01:52.436 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2024-09-15T21:01:52.436 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2024-09-15T21:01:52.436 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:52.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-15T21:01:52.579 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-15T21:01:52.579 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2024-09-15T21:01:52.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2024-09-15T21:01:52.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:142 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.991578048999997, 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60625672200000003, 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.580 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 142 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582864+0000", 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.582867+0000", 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3740000000000001e-06 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583487+0000", 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00062026900000000001 2024-09-15T21:01:52.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583517+0000", 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9711000000000001e-05 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583534+0000", 2024-09-15T21:01:52.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583673+0000", 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013930799999999999 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583987+0000", 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00031381600000000002 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.585772+0000", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001785046 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.591440+0000", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056679770000000003 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189000+0000", 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.59755970599999997 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189121+0000", 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000121323 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:143 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.990870692000001, 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60580102899999999, 2024-09-15T21:01:52.588 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 143 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583572+0000", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.583573+0000", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584094+0000", 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00052068600000000002 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584112+0000", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584122+0000", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.7329999999999998e-06 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587442+0000", 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0033205719999999999 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.587680+0000", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00023744700000000001 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.590202+0000", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025220780000000001 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.592136+0000", 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019345059999999999 2024-09-15T21:01:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189282+0000", 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.597146014 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.189373+0000", 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0708e-05 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:144 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.990297549000005, 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60840782599999998, 2024-09-15T21:01:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 144 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584145+0000", 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584146+0000", 2024-09-15T21:01:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.516e-06 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584156+0000", 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6660000000000003e-06 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.584174+0000", 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7635999999999999e-05 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684348+0000", 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100174129 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784647+0000", 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100299521 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.784920+0000", 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000272589 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.786132+0000", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001212177 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.887007+0000", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100874637 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192327+0000", 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30531983000000001 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.192553+0000", 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022612500000000001 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:145 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.890055607999997, 2024-09-15T21:01:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62539599999999995, 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 145 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684387+0000", 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684389+0000", 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.64e-06 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684397+0000", 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8119999999999997e-06 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684417+0000", 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9572000000000002e-05 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.684428+0000", 2024-09-15T21:01:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1012999999999999e-05 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886782+0000", 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20235414500000001 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.886921+0000", 2024-09-15T21:01:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013896399999999999 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193325+0000", 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.306403863 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.193486+0000", 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000160741 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.309783+0000", 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.11629725 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:154 3.8s0 3:15433346:::smithi02732947-2:head [copy-from ver 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.783797308000004, 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.80461004300000005, 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 154 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790645+0000", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790648+0000", 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9960000000000002e-06 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790661+0000", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3106e-05 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790700+0000", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8194000000000001e-05 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790718+0000", 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8391999999999999e-05 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790864+0000", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014588000000000001 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791137+0000", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027360200000000001 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.312842+0000", 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52170422900000002 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314538+0000", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001696468 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595051+0000", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.280513175 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.595255+0000", 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020400099999999999 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:155 3.8s0 3:15433346:::smithi02732947-2:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 87.783684872999999, 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52377280199999998, 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 155 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790758+0000", 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790759+0000", 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.587e-06 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790769+0000", 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5829999999999996e-06 2024-09-15T21:01:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790791+0000", 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2242000000000002e-05 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.790808+0000", 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6569e-05 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791598+0000", 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000790454 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for blocked object", 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:24.791740+0000", 2024-09-15T21:01:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014175799999999999 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.313956+0000", 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.52221568299999999 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314260+0000", 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030485000000000001 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:25.314530+0000", 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00027007600000000001 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:500 3.es0 3:7cf15e4b:::smithi02732947-48:head [copy-from ver 8] snapc 12=[12,11,10,e] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.933526381999997, 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.72001296999999997, 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 500 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640916+0000", 2024-09-15T21:01:52.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640919+0000", 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3100000000000001e-06 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640934+0000", 2024-09-15T21:01:52.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.491e-05 2024-09-15T21:01:52.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640969+0000", 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4686e-05 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.640986+0000", 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7048999999999999e-05 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641119+0000", 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013317 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.641136+0000", 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6558999999999998e-05 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.845985+0000", 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20484875999999999 2024-09-15T21:01:52.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:56.846249+0000", 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026397900000000001 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.056421+0000", 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.210172042 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.059407+0000", 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029868300000000002 2024-09-15T21:01:52.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360789+0000", 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.30138199900000001 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:00:57.360929+0000", 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013967600000000001 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:832 3.8s0 3:15433346:::smithi02732947-2:head [append 0~720896] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.879583781000001, 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.57907397599999999, 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 832 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694859+0000", 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.694862+0000", 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2059999999999999e-06 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696156+0000", 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012936379999999999 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696190+0000", 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4125000000000003e-05 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696208+0000", 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8442000000000001e-05 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696374+0000", 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000166133 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op must wait for map", 2024-09-15T21:01:52.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.696390+0000", 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5885000000000001e-05 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749483+0000", 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053093057999999999 2024-09-15T21:01:52.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.749694+0000", 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021056300000000001 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.752168+0000", 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024746529999999998 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763652+0000", 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011483446 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273791+0000", 2024-09-15T21:01:52.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51013918300000005 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.273933+0000", 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014164399999999999 2024-09-15T21:01:52.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:833 3.8s0 3:15433346:::smithi02732947-2:head [append 0~622592] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:52.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.874485419999999, 2024-09-15T21:01:52.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.67449799700000002, 2024-09-15T21:01:52.645 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 833 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.646 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699957+0000", 2024-09-15T21:01:52.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.699960+0000", 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.88e-06 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701223+0000", 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0012632870000000001 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701257+0000", 2024-09-15T21:01:52.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4004999999999997e-05 2024-09-15T21:01:52.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.701280+0000", 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3144999999999999e-05 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.754820+0000", 2024-09-15T21:01:52.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.053539760999999998 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.755101+0000", 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028043799999999999 2024-09-15T21:01:52.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.758090+0000", 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002989317 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.763734+0000", 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0056437930000000002 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374313+0000", 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.61057963000000004 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374455+0000", 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141741 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:834 3.8s0 3:15433346:::smithi02732947-2:head [append 0~688128] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.868544072999999, 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66884350100000001, 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 834 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.659 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705898+0000", 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.705901+0000", 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1399999999999998e-06 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707380+0000", 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014797160000000001 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707409+0000", 2024-09-15T21:01:52.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8951000000000001e-05 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.707431+0000", 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1412000000000001e-05 2024-09-15T21:01:52.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759507+0000", 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.052076560000000001 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.759651+0000", 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014353099999999999 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.762259+0000", 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002607786 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.864069+0000", 2024-09-15T21:01:52.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101810885 2024-09-15T21:01:52.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374639+0000", 2024-09-15T21:01:52.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51056974700000002 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374742+0000", 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000102773 2024-09-15T21:01:52.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:835 3.8s0 3:15433346:::smithi02732947-2:head [append 0~245760] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.866345781, 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.667032505, 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.671 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 835 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708097+0000", 2024-09-15T21:01:52.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708099+0000", 2024-09-15T21:01:52.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.553e-06 2024-09-15T21:01:52.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708461+0000", 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036148899999999999 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708494+0000", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3674999999999999e-05 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708517+0000", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2084999999999999e-05 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967220+0000", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.25870379199999999 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.967394+0000", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017396100000000001 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.968720+0000", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325737 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077387+0000", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10866701600000001 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.374989+0000", 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.29760241199999998 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375129+0000", 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013978499999999999 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:836 3.8s0 3:15433346:::smithi02732947-2:head [setxattr _header (56) in=7b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:52.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.865888212000002, 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.66703066, 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 836 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708554+0000", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708556+0000", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8670000000000001e-06 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708565+0000", 2024-09-15T21:01:52.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5269999999999998e-06 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708590+0000", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.5001e-05 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708606+0000", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6776e-05 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.969953+0000", 2024-09-15T21:01:52.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26134606799999999 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.970101+0000", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014809999999999999 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971049+0000", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000948678 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077391+0000", 2024-09-15T21:01:52.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.10634163100000001 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375445+0000", 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298053911 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375585+0000", 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014010100000000001 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:837 3.8s0 3:15433346:::smithi02732947-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.865809447, 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.87430144200000004, 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 837 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708633+0000", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708634+0000", 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.364e-06 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708641+0000", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9229999999999996e-06 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708665+0000", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3515000000000001e-05 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.708681+0000", 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.6395000000000001e-05 2024-09-15T21:01:52.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971593+0000", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.26291159200000003 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971759+0000", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165966 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378011+0000", 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.406252055 2024-09-15T21:01:52.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.378267+0000", 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025595099999999998 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.582935+0000", 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.20466768099999999 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:844 3.0s0 3:0196c8aa:::smithi02732947-4:head [create,setxattr lnO\\b_Vrmg\\YoalgS_ap]_oaNZQ[WWtVp\\Pqajab (15856) in=40b,setxattr lk^eORZ`gVcm``\\pr^dth`]NqmVUQkUObOacmWfh (699) in=40b,setxattr alfebafpqnNmSoe_jrs[jlogjdclQl]SfW^_opiR (911) in=40b,setxattr ]tVYcbfsnki]TcRbjSXlkPP\\URmibcZ (559) in=31b,setxattr cS_QsbNZRaNftQNafpmOXrSq[QTT^a`bjr^RjcXe (19716) in=40b,setxattr lWQarRah[_Ri`aaaYU\\YttQj[iZtlmkV]_giaXU] (1119) in=40b,setxattr afP^ZXR]oT[nYcXcXtmS^YhRRO`ah[[igtU\\maam (3576) in=40b,setxattr lfUpnr]dS`jeYdSeReTo^[[scdZRhaXsnpNagRV] (417) in=40b,setxattr oRZflPsTY\\iqWn_qits]aghfipZmOmndmY\\_fdsm (453) in=40b,setxattr cPXrXtr[heq_Y[rbnflZtdiTN[ejbhn]QUcpclY[ (10312) in=40b,setxattr t_S^QjNpSe[mtQWcjpeoQbsQWZqNlnhb`jXVNbkc (1647) in=40b,setxattr QhqWq\\srNqhaWrNUTpj\\lra]SNOZnScUZTX`NXWh (1717) in=40b,setxattr [VXbj[Sc_ttR`ab\\tdcTOlesaQOosdTTQnNtqnS\\ (1329) in=40b,setxattr kOPlXRXq\\neZk[`Qch\\fXP`]t`NWhTlj_XPZhlSY (4560) in=40b,setxattr P^eT`anhlNcnlqOfQsV^QOeZhkX^VbfPm`aNNkSN (1401) in=40b,setxattr gTqoYWiXq`b\\`jd\\V\\URmtOrZernYT_aSTkmR_lV (1715) in=40b,setxattr immbP]\\[qVq`fc\\]bYS\\lmgX[Sk_nfrSS_pTheVl (17540) in=40b,setxattr dRRpo\\[dNWhVPhqhmg[pqo_][oUNQgkOpkpaRPhS (1017) in=40b,setxattr [\\\\ofp^gsRlqNh]W^sW`[[qhU_rb\\qdg\\cffde_g (13964) in=40b,setxattr \\d[kjqp`[PR] (559) in=12b] snapc 1d=[1d,1c,1b,1a,19,18,16] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 31.834067723, 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.63608215899999998, 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 844 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740375+0000", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740405+0000", 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0065000000000001e-05 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740634+0000", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000229232 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740675+0000", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0556999999999998e-05 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.740697+0000", 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2138999999999999e-05 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.971827+0000", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.23113030700000001 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.972183+0000", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00035558600000000002 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:20.974826+0000", 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0026435999999999999 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.077486+0000", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102659311 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.375591+0000", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.298105283 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:21.376457+0000", 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00086607899999999996 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:971 3.es0 3:7cf15e4b:::smithi02732947-48:head [append 0~98304] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.695922+0000", 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.878520503000001, 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60437397599999998, 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 971 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.695922+0000", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.695922+0000", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.695924+0000", 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7570000000000001e-06 2024-09-15T21:01:52.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696153+0000", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022955499999999999 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696171+0000", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7604e-05 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696179+0000", 2024-09-15T21:01:52.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3270000000000001e-06 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696634+0000", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00045503299999999998 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696781+0000", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014690600000000001 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.697390+0000", 2024-09-15T21:01:52.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000609196 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.798870+0000", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101479342 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.300215+0000", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50134500199999998 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.300296+0000", 2024-09-15T21:01:52.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1254000000000005e-05 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:972 3.es0 3:7cf15e4b:::smithi02732947-48:head [setxattr _header (57) in=7b] snapc 22=[22,21,1f,1b,16] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.696199+0000", 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.878243446999999, 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.60466231199999998, 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.700 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 972 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696199+0000", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696199+0000", 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696200+0000", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.44e-07 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696203+0000", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0790000000000001e-06 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696209+0000", 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1090000000000003e-06 2024-09-15T21:01:52.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696213+0000", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.42e-06 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.797933+0000", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.101719804 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.798033+0000", 2024-09-15T21:01:52.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9888000000000003e-05 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_started", 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.798522+0000", 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00048861399999999995 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_committed", 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.899153+0000", 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100631126 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2024-09-15T21:01:52.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.300723+0000", 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.401570551 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.300861+0000", 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013817699999999999 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:973 3.es0 3:7cf15e4b:::smithi02732947-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:37.696238+0000", 2024-09-15T21:01:52.705 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.878204754, 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.76441526999999998, 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 973 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696238+0000", 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696238+0000", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696238+0000", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3000000000000001e-07 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696241+0000", 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:01:52.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696247+0000", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1539999999999999e-06 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.696251+0000", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4870000000000004e-06 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.798909+0000", 2024-09-15T21:01:52.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.102657793 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for rw locks", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:37.799030+0000", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000120736 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.301503+0000", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.50247313699999996 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.301633+0000", 2024-09-15T21:01:52.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012989800000000001 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.460653+0000", 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.15902008000000001 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:982 3.8s0 3:15433346:::smithi02732947-2:head [read 0~5537792,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.163471+0000", 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.410971976000001, 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51544680700000001, 2024-09-15T21:01:52.710 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 982 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163471+0000", 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163471+0000", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163473+0000", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6960000000000001e-06 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163484+0000", 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0978000000000001e-05 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163503+0000", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8739000000000001e-05 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163513+0000", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6239999999999997e-06 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.558770+0000", 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.39525736900000003 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.559055+0000", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028550500000000003 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.678917+0000", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.119861896 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:983 3.8s0 3:15433346:::smithi02732947-2:head [read 1167597~4370195,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.163528+0000", 2024-09-15T21:01:52.714 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.410914375000001, 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.51926697399999999, 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 983 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163528+0000", 2024-09-15T21:01:52.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163528+0000", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163529+0000", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0999999999999998e-07 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163533+0000", 2024-09-15T21:01:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6369999999999998e-06 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163541+0000", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9619999999999993e-06 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163547+0000", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5589999999999996e-06 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.659185+0000", 2024-09-15T21:01:52.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.49563854099999999 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.659393+0000", 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020719900000000001 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.682795+0000", 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.023402466 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4641.0:984 3.8s0 3:15433346:::smithi02732947-2:head [read 2712547~2825245,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2024-09-15T21:01:38.163554+0000", 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.410888722999999, 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.62431432099999995, 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "continuous": false, 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4641", 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "172.21.15.27:0/545227388", 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 984 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163554+0000", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163554+0000", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163554+0000", 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500000000000001e-07 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163556+0000", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1840000000000002e-06 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.163562+0000", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3360000000000002e-06 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.263655+0000", 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.100093561 2024-09-15T21:01:52.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.660377+0000", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.396722193 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.660529+0000", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015188300000000001 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2024-09-15T21:01:38.787868+0000", 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.12733878900000001 2024-09-15T21:01:52.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.723 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-15T21:01:52.723 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-15T21:01:52.723 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-15T21:01:52.856 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T21:01:52.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T21:01:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T21:01:53.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T21:01:54.876 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T21:01:54.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T21:01:55.887 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T21:01:55.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T21:01:56.905 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T21:01:56.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T21:01:57.907 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T21:01:57.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T21:01:58.927 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T21:01:58.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T21:01:59.936 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T21:01:59.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T21:02:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T21:02:00.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T21:02:01.948 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T21:02:01.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T21:02:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T21:02:02.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T21:02:03.965 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T21:02:03.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T21:02:04.981 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T21:02:05.008 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T21:02:05.984 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T21:02:06.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T21:02:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T21:02:07.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-15T21:02:08.002 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.8 to 1 (10000) 2024-09-15T21:02:08.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-15T21:02:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-15T21:02:09.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-15T21:02:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.9 to 1 (10000) 2024-09-15T21:02:10.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-15T21:02:11.037 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-15T21:02:11.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-15T21:02:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.10 to 1 (10000) 2024-09-15T21:02:12.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-15T21:02:13.047 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-15T21:02:13.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-15T21:02:14.056 INFO:teuthology.orchestra.run.smithi027.stderr:reweighted osd.11 to 1 (10000) 2024-09-15T21:02:14.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-15T21:02:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-15T21:02:15.095 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T21:02:15.095 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T21:02:15.095 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T21:02:15.095 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T21:02:15.095 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:02:15.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:02:15.413 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:15.414 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":139,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:02:15.061701+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T21:01:16.623312+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:03.908437+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:10.056446+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.815936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.317862+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:58.417589+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:06.653717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:07.037212+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:24.672718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:01.773094+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:00.521472+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:00:55.547591+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:15.906985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":3},{"begin":32,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:02:15.432 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:02:15.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:02:15.743 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:15.743 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":139,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:02:15.061701+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T21:01:16.623312+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:03.908437+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:10.056446+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.815936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.317862+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:58.417589+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:06.653717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:07.037212+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:24.672718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:01.773094+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:00.521472+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:00:55.547591+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:15.906985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":3},{"begin":32,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:02:15.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:02:15.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:02:15.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:02:15.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:02:15.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:02:15.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:02:15.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:02:15.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:02:15.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-15T21:02:15.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-15T21:02:15.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-15T21:02:15.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-15T21:02:16.120 INFO:teuthology.orchestra.run.smithi027.stdout:77309411360 2024-09-15T21:02:16.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T21:02:16.121 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:16.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:02:16.131 INFO:teuthology.orchestra.run.smithi027.stdout:73014444062 2024-09-15T21:02:16.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:02:16.144 INFO:teuthology.orchestra.run.smithi027.stdout:73014444064 2024-09-15T21:02:16.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:02:16.156 INFO:teuthology.orchestra.run.smithi027.stdout:77309411358 2024-09-15T21:02:16.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:02:16.165 INFO:teuthology.orchestra.run.smithi027.stdout:77309411359 2024-09-15T21:02:16.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T21:02:16.169 INFO:teuthology.orchestra.run.smithi027.stdout:73014444064 2024-09-15T21:02:16.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:02:16.170 INFO:teuthology.orchestra.run.smithi027.stdout:81604378656 2024-09-15T21:02:16.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:02:16.186 INFO:teuthology.orchestra.run.smithi027.stdout:77309411358 2024-09-15T21:02:16.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T21:02:16.190 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:16.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:02:16.204 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:16.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:02:16.207 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:16.208 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T21:02:16.808 INFO:teuthology.orchestra.run.smithi027.stdout:73014444062 2024-09-15T21:02:16.846 INFO:teuthology.orchestra.run.smithi027.stdout:73014444061 2024-09-15T21:02:16.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444062 for osd.0 2024-09-15T21:02:16.867 INFO:teuthology.orchestra.run.smithi027.stdout:73014444061 2024-09-15T21:02:16.898 INFO:teuthology.orchestra.run.smithi027.stdout:77309411358 2024-09-15T21:02:16.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444061 for osd.1 2024-09-15T21:02:16.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444062 got 73014444061 for osd.4 2024-09-15T21:02:16.927 INFO:teuthology.orchestra.run.smithi027.stdout:81604378655 2024-09-15T21:02:16.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411359 got 77309411358 for osd.10 2024-09-15T21:02:16.942 INFO:teuthology.orchestra.run.smithi027.stdout:77309411359 2024-09-15T21:02:16.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378656 got 81604378655 for osd.2 2024-09-15T21:02:16.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411360 got 77309411359 for osd.11 2024-09-15T21:02:16.985 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:17.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444064 got 73014444063 for osd.3 2024-09-15T21:02:17.011 INFO:teuthology.orchestra.run.smithi027.stdout:77309411357 2024-09-15T21:02:17.032 INFO:teuthology.orchestra.run.smithi027.stdout:77309411357 2024-09-15T21:02:17.034 INFO:teuthology.orchestra.run.smithi027.stdout:73014444062 2024-09-15T21:02:17.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411358 got 77309411357 for osd.8 2024-09-15T21:02:17.049 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:17.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444062 for osd.5 2024-09-15T21:02:17.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411358 got 77309411357 for osd.7 2024-09-15T21:02:17.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444064 got 73014444063 for osd.6 2024-09-15T21:02:17.076 INFO:teuthology.orchestra.run.smithi027.stdout:73014444062 2024-09-15T21:02:17.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444062 for osd.9 2024-09-15T21:02:17.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:02:17.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:02:17.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:02:17.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-15T21:02:17.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:02:17.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-15T21:02:18.004 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:02:18.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-15T21:02:18.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:02:18.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:02:18.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:02:18.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-15T21:02:18.398 INFO:teuthology.orchestra.run.smithi027.stdout:73014444064 2024-09-15T21:02:18.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444064 for osd.0 2024-09-15T21:02:18.461 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.484 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:18.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444063 for osd.1 2024-09-15T21:02:18.526 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.595 INFO:teuthology.orchestra.run.smithi027.stdout:73014444063 2024-09-15T21:02:18.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444062 got 73014444063 for osd.4 2024-09-15T21:02:18.649 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.682 INFO:teuthology.orchestra.run.smithi027.stdout:77309411359 2024-09-15T21:02:18.683 INFO:teuthology.orchestra.run.smithi027.stdout:77309411359 2024-09-15T21:02:18.683 INFO:teuthology.orchestra.run.smithi027.stdout:81604378656 2024-09-15T21:02:18.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378656 got 81604378656 for osd.2 2024-09-15T21:02:18.711 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.712 INFO:teuthology.orchestra.run.smithi027.stdout:77309411360 2024-09-15T21:02:18.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411359 got 77309411359 for osd.10 2024-09-15T21:02:18.715 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411358 got 77309411359 for osd.8 2024-09-15T21:02:18.726 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411360 got 77309411360 for osd.11 2024-09-15T21:02:18.739 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.775 INFO:teuthology.orchestra.run.smithi027.stdout:77309411359 2024-09-15T21:02:18.783 INFO:teuthology.orchestra.run.smithi027.stdout:73014444064 2024-09-15T21:02:18.785 INFO:teuthology.orchestra.run.smithi027.stdout:73014444065 2024-09-15T21:02:18.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411358 got 77309411359 for osd.7 2024-09-15T21:02:18.805 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.806 INFO:teuthology.orchestra.run.smithi027.stdout:73014444065 2024-09-15T21:02:18.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444064 got 73014444065 for osd.3 2024-09-15T21:02:18.813 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444064 for osd.5 2024-09-15T21:02:18.814 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444064 got 73014444065 for osd.6 2024-09-15T21:02:18.825 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.870 INFO:teuthology.orchestra.run.smithi027.stdout:73014444064 2024-09-15T21:02:18.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444063 got 73014444064 for osd.9 2024-09-15T21:02:18.887 DEBUG:teuthology.parallel:result is None 2024-09-15T21:02:18.887 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T21:02:18.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:19.231 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:19.232 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:19.248 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-15T21:02:19.109671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484116,"kb_used_data":21028,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977964,"statfs":{"total":1159641169920,"available":1159145435136,"internally_reserved":0,"allocated":21532672,"data_stored":13777966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.694824"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.412882+0000","last_change":"2024-09-15T21:02:11.601079+0000","last_active":"2024-09-15T21:02:15.412882+0000","last_peered":"2024-09-15T21:02:15.412882+0000","last_clean":"2024-09-15T21:02:15.412882+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:15.412882+0000","last_undegraded":"2024-09-15T21:02:15.412882+0000","last_fullsized":"2024-09-15T21:02:15.412882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:11.600924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:11.600924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:27.668475+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072500+0000","last_change":"2024-09-15T21:02:00.442851+0000","last_active":"2024-09-15T21:02:15.072500+0000","last_peered":"2024-09-15T21:02:15.072500+0000","last_clean":"2024-09-15T21:02:15.072500+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:15.072500+0000","last_undegraded":"2024-09-15T21:02:15.072500+0000","last_fullsized":"2024-09-15T21:02:15.072500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:00.442669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:00.442669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:24.507074+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073236+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:02:15.073236+0000","last_peered":"2024-09-15T21:02:15.073236+0000","last_clean":"2024-09-15T21:02:15.073236+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:15.073236+0000","last_undegraded":"2024-09-15T21:02:15.073236+0000","last_fullsized":"2024-09-15T21:02:15.073236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072470+0000","last_change":"2024-09-15T21:01:57.184317+0000","last_active":"2024-09-15T21:02:15.072470+0000","last_peered":"2024-09-15T21:02:15.072470+0000","last_clean":"2024-09-15T21:02:15.072470+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:15.072470+0000","last_undegraded":"2024-09-15T21:02:15.072470+0000","last_fullsized":"2024-09-15T21:02:15.072470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:57.184117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:57.184117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.869457+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":275,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073377+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:02:15.073377+0000","last_peered":"2024-09-15T21:02:15.073377+0000","last_clean":"2024-09-15T21:02:15.073377+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:15.073377+0000","last_undegraded":"2024-09-15T21:02:15.073377+0000","last_fullsized":"2024-09-15T21:02:15.073377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:16.055967+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:02:16.055967+0000","last_peered":"2024-09-15T21:02:16.055967+0000","last_clean":"2024-09-15T21:02:16.055967+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:16.055967+0000","last_undegraded":"2024-09-15T21:02:16.055967+0000","last_fullsized":"2024-09-15T21:02:16.055967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411385+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:15.411385+0000","last_peered":"2024-09-15T21:02:15.411385+0000","last_clean":"2024-09-15T21:02:15.411385+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:15.411385+0000","last_undegraded":"2024-09-15T21:02:15.411385+0000","last_fullsized":"2024-09-15T21:02:15.411385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":259,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073404+0000","last_change":"2024-09-15T21:01:48.868446+0000","last_active":"2024-09-15T21:02:15.073404+0000","last_peered":"2024-09-15T21:02:15.073404+0000","last_clean":"2024-09-15T21:02:15.073404+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:15.073404+0000","last_undegraded":"2024-09-15T21:02:15.073404+0000","last_fullsized":"2024-09-15T21:02:15.073404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:01:48.868291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:48.868291+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.628848+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":369,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411675+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:02:15.411675+0000","last_peered":"2024-09-15T21:02:15.411675+0000","last_clean":"2024-09-15T21:02:15.411675+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:15.411675+0000","last_undegraded":"2024-09-15T21:02:15.411675+0000","last_fullsized":"2024-09-15T21:02:15.411675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55580,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316260,"statfs":{"total":96636764160,"available":96579850240,"internally_reserved":0,"allocated":2232320,"data_stored":1583814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411360,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2293760,"data_stored":1645254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32852,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55060,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316780,"statfs":{"total":96636764160,"available":96580382720,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444063,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:19.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:19.544 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:19.545 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:19.561 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-15T21:02:19.416732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484116,"kb_used_data":21028,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977964,"statfs":{"total":1159641169920,"available":1159145435136,"internally_reserved":0,"allocated":21532672,"data_stored":13777966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360317"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.412882+0000","last_change":"2024-09-15T21:02:11.601079+0000","last_active":"2024-09-15T21:02:15.412882+0000","last_peered":"2024-09-15T21:02:15.412882+0000","last_clean":"2024-09-15T21:02:15.412882+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:15.412882+0000","last_undegraded":"2024-09-15T21:02:15.412882+0000","last_fullsized":"2024-09-15T21:02:15.412882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:11.600924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:11.600924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:27.668475+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072500+0000","last_change":"2024-09-15T21:02:00.442851+0000","last_active":"2024-09-15T21:02:15.072500+0000","last_peered":"2024-09-15T21:02:15.072500+0000","last_clean":"2024-09-15T21:02:15.072500+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:15.072500+0000","last_undegraded":"2024-09-15T21:02:15.072500+0000","last_fullsized":"2024-09-15T21:02:15.072500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:00.442669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:00.442669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:24.507074+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073236+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:02:15.073236+0000","last_peered":"2024-09-15T21:02:15.073236+0000","last_clean":"2024-09-15T21:02:15.073236+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:15.073236+0000","last_undegraded":"2024-09-15T21:02:15.073236+0000","last_fullsized":"2024-09-15T21:02:15.073236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072470+0000","last_change":"2024-09-15T21:01:57.184317+0000","last_active":"2024-09-15T21:02:15.072470+0000","last_peered":"2024-09-15T21:02:15.072470+0000","last_clean":"2024-09-15T21:02:15.072470+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:15.072470+0000","last_undegraded":"2024-09-15T21:02:15.072470+0000","last_fullsized":"2024-09-15T21:02:15.072470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:57.184117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:57.184117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.869457+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":275,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073377+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:02:15.073377+0000","last_peered":"2024-09-15T21:02:15.073377+0000","last_clean":"2024-09-15T21:02:15.073377+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:15.073377+0000","last_undegraded":"2024-09-15T21:02:15.073377+0000","last_fullsized":"2024-09-15T21:02:15.073377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:16.055967+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:02:16.055967+0000","last_peered":"2024-09-15T21:02:16.055967+0000","last_clean":"2024-09-15T21:02:16.055967+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:16.055967+0000","last_undegraded":"2024-09-15T21:02:16.055967+0000","last_fullsized":"2024-09-15T21:02:16.055967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411385+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:15.411385+0000","last_peered":"2024-09-15T21:02:15.411385+0000","last_clean":"2024-09-15T21:02:15.411385+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:15.411385+0000","last_undegraded":"2024-09-15T21:02:15.411385+0000","last_fullsized":"2024-09-15T21:02:15.411385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":259,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073404+0000","last_change":"2024-09-15T21:01:48.868446+0000","last_active":"2024-09-15T21:02:15.073404+0000","last_peered":"2024-09-15T21:02:15.073404+0000","last_clean":"2024-09-15T21:02:15.073404+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:15.073404+0000","last_undegraded":"2024-09-15T21:02:15.073404+0000","last_fullsized":"2024-09-15T21:02:15.073404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:01:48.868291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:48.868291+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.628848+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":369,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411675+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:02:15.411675+0000","last_peered":"2024-09-15T21:02:15.411675+0000","last_clean":"2024-09-15T21:02:15.411675+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:15.411675+0000","last_undegraded":"2024-09-15T21:02:15.411675+0000","last_fullsized":"2024-09-15T21:02:15.411675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55580,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316260,"statfs":{"total":96636764160,"available":96579850240,"internally_reserved":0,"allocated":2232320,"data_stored":1583814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411360,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2293760,"data_stored":1645254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32852,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55060,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316780,"statfs":{"total":96636764160,"available":96580382720,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444063,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:19.562 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T21:02:19.562 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T21:02:19.887 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:02:19.893 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:02:19.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:20.196 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:20.214 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-15T21:02:19.416732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484116,"kb_used_data":21028,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977964,"statfs":{"total":1159641169920,"available":1159145435136,"internally_reserved":0,"allocated":21532672,"data_stored":13777966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360317"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.412882+0000","last_change":"2024-09-15T21:02:11.601079+0000","last_active":"2024-09-15T21:02:15.412882+0000","last_peered":"2024-09-15T21:02:15.412882+0000","last_clean":"2024-09-15T21:02:15.412882+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:15.412882+0000","last_undegraded":"2024-09-15T21:02:15.412882+0000","last_fullsized":"2024-09-15T21:02:15.412882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:11.600924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:11.600924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:27.668475+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072500+0000","last_change":"2024-09-15T21:02:00.442851+0000","last_active":"2024-09-15T21:02:15.072500+0000","last_peered":"2024-09-15T21:02:15.072500+0000","last_clean":"2024-09-15T21:02:15.072500+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:15.072500+0000","last_undegraded":"2024-09-15T21:02:15.072500+0000","last_fullsized":"2024-09-15T21:02:15.072500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:00.442669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:00.442669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:24.507074+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073236+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:02:15.073236+0000","last_peered":"2024-09-15T21:02:15.073236+0000","last_clean":"2024-09-15T21:02:15.073236+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:15.073236+0000","last_undegraded":"2024-09-15T21:02:15.073236+0000","last_fullsized":"2024-09-15T21:02:15.073236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072470+0000","last_change":"2024-09-15T21:01:57.184317+0000","last_active":"2024-09-15T21:02:15.072470+0000","last_peered":"2024-09-15T21:02:15.072470+0000","last_clean":"2024-09-15T21:02:15.072470+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:15.072470+0000","last_undegraded":"2024-09-15T21:02:15.072470+0000","last_fullsized":"2024-09-15T21:02:15.072470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:57.184117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:57.184117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.869457+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":275,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073377+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:02:15.073377+0000","last_peered":"2024-09-15T21:02:15.073377+0000","last_clean":"2024-09-15T21:02:15.073377+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:15.073377+0000","last_undegraded":"2024-09-15T21:02:15.073377+0000","last_fullsized":"2024-09-15T21:02:15.073377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:16.055967+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:02:16.055967+0000","last_peered":"2024-09-15T21:02:16.055967+0000","last_clean":"2024-09-15T21:02:16.055967+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:16.055967+0000","last_undegraded":"2024-09-15T21:02:16.055967+0000","last_fullsized":"2024-09-15T21:02:16.055967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411385+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:15.411385+0000","last_peered":"2024-09-15T21:02:15.411385+0000","last_clean":"2024-09-15T21:02:15.411385+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:15.411385+0000","last_undegraded":"2024-09-15T21:02:15.411385+0000","last_fullsized":"2024-09-15T21:02:15.411385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":259,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073404+0000","last_change":"2024-09-15T21:01:48.868446+0000","last_active":"2024-09-15T21:02:15.073404+0000","last_peered":"2024-09-15T21:02:15.073404+0000","last_clean":"2024-09-15T21:02:15.073404+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:15.073404+0000","last_undegraded":"2024-09-15T21:02:15.073404+0000","last_fullsized":"2024-09-15T21:02:15.073404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:01:48.868291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:48.868291+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.628848+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":369,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411675+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:02:15.411675+0000","last_peered":"2024-09-15T21:02:15.411675+0000","last_clean":"2024-09-15T21:02:15.411675+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:15.411675+0000","last_undegraded":"2024-09-15T21:02:15.411675+0000","last_fullsized":"2024-09-15T21:02:15.411675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55580,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316260,"statfs":{"total":96636764160,"available":96579850240,"internally_reserved":0,"allocated":2232320,"data_stored":1583814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411360,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2293760,"data_stored":1645254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32852,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55060,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316780,"statfs":{"total":96636764160,"available":96580382720,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444063,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:20.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:20.515 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:20.532 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-15T21:02:19.416732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484116,"kb_used_data":21028,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977964,"statfs":{"total":1159641169920,"available":1159145435136,"internally_reserved":0,"allocated":21532672,"data_stored":13777966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360317"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.412882+0000","last_change":"2024-09-15T21:02:11.601079+0000","last_active":"2024-09-15T21:02:15.412882+0000","last_peered":"2024-09-15T21:02:15.412882+0000","last_clean":"2024-09-15T21:02:15.412882+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:15.412882+0000","last_undegraded":"2024-09-15T21:02:15.412882+0000","last_fullsized":"2024-09-15T21:02:15.412882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:11.600924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:11.600924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:27.668475+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072500+0000","last_change":"2024-09-15T21:02:00.442851+0000","last_active":"2024-09-15T21:02:15.072500+0000","last_peered":"2024-09-15T21:02:15.072500+0000","last_clean":"2024-09-15T21:02:15.072500+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:15.072500+0000","last_undegraded":"2024-09-15T21:02:15.072500+0000","last_fullsized":"2024-09-15T21:02:15.072500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:00.442669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:00.442669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:24.507074+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073236+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:02:15.073236+0000","last_peered":"2024-09-15T21:02:15.073236+0000","last_clean":"2024-09-15T21:02:15.073236+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:15.073236+0000","last_undegraded":"2024-09-15T21:02:15.073236+0000","last_fullsized":"2024-09-15T21:02:15.073236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072470+0000","last_change":"2024-09-15T21:01:57.184317+0000","last_active":"2024-09-15T21:02:15.072470+0000","last_peered":"2024-09-15T21:02:15.072470+0000","last_clean":"2024-09-15T21:02:15.072470+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:15.072470+0000","last_undegraded":"2024-09-15T21:02:15.072470+0000","last_fullsized":"2024-09-15T21:02:15.072470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:57.184117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:57.184117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.869457+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":275,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073377+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:02:15.073377+0000","last_peered":"2024-09-15T21:02:15.073377+0000","last_clean":"2024-09-15T21:02:15.073377+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:15.073377+0000","last_undegraded":"2024-09-15T21:02:15.073377+0000","last_fullsized":"2024-09-15T21:02:15.073377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:16.055967+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:02:16.055967+0000","last_peered":"2024-09-15T21:02:16.055967+0000","last_clean":"2024-09-15T21:02:16.055967+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:16.055967+0000","last_undegraded":"2024-09-15T21:02:16.055967+0000","last_fullsized":"2024-09-15T21:02:16.055967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411385+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:15.411385+0000","last_peered":"2024-09-15T21:02:15.411385+0000","last_clean":"2024-09-15T21:02:15.411385+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:15.411385+0000","last_undegraded":"2024-09-15T21:02:15.411385+0000","last_fullsized":"2024-09-15T21:02:15.411385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":259,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073404+0000","last_change":"2024-09-15T21:01:48.868446+0000","last_active":"2024-09-15T21:02:15.073404+0000","last_peered":"2024-09-15T21:02:15.073404+0000","last_clean":"2024-09-15T21:02:15.073404+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:15.073404+0000","last_undegraded":"2024-09-15T21:02:15.073404+0000","last_fullsized":"2024-09-15T21:02:15.073404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:01:48.868291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:48.868291+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.628848+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":369,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411675+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:02:15.411675+0000","last_peered":"2024-09-15T21:02:15.411675+0000","last_clean":"2024-09-15T21:02:15.411675+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:15.411675+0000","last_undegraded":"2024-09-15T21:02:15.411675+0000","last_fullsized":"2024-09-15T21:02:15.411675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55580,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316260,"statfs":{"total":96636764160,"available":96579850240,"internally_reserved":0,"allocated":2232320,"data_stored":1583814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411360,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2293760,"data_stored":1645254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32852,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55060,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316780,"statfs":{"total":96636764160,"available":96580382720,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444063,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:20.533 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:02:20.534 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:20.828 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:20.828 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:20.845 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-15T21:02:19.416732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484116,"kb_used_data":21028,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977964,"statfs":{"total":1159641169920,"available":1159145435136,"internally_reserved":0,"allocated":21532672,"data_stored":13777966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360317"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":251,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.412882+0000","last_change":"2024-09-15T21:02:11.601079+0000","last_active":"2024-09-15T21:02:15.412882+0000","last_peered":"2024-09-15T21:02:15.412882+0000","last_clean":"2024-09-15T21:02:15.412882+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:15.412882+0000","last_undegraded":"2024-09-15T21:02:15.412882+0000","last_fullsized":"2024-09-15T21:02:15.412882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:11.600924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:11.600924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:27.668475+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072500+0000","last_change":"2024-09-15T21:02:00.442851+0000","last_active":"2024-09-15T21:02:15.072500+0000","last_peered":"2024-09-15T21:02:15.072500+0000","last_clean":"2024-09-15T21:02:15.072500+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:15.072500+0000","last_undegraded":"2024-09-15T21:02:15.072500+0000","last_fullsized":"2024-09-15T21:02:15.072500+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:00.442669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:00.442669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:24.507074+0000","scrub_duration":511,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":256,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073236+0000","last_change":"2024-09-15T21:01:16.726365+0000","last_active":"2024-09-15T21:02:15.073236+0000","last_peered":"2024-09-15T21:02:15.073236+0000","last_clean":"2024-09-15T21:02:15.073236+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:15.073236+0000","last_undegraded":"2024-09-15T21:02:15.073236+0000","last_fullsized":"2024-09-15T21:02:15.073236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:16.726193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:16.726193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:39.053582+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.072470+0000","last_change":"2024-09-15T21:01:57.184317+0000","last_active":"2024-09-15T21:02:15.072470+0000","last_peered":"2024-09-15T21:02:15.072470+0000","last_clean":"2024-09-15T21:02:15.072470+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:15.072470+0000","last_undegraded":"2024-09-15T21:02:15.072470+0000","last_fullsized":"2024-09-15T21:02:15.072470+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:01:57.184117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:57.184117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:58.869457+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"24'2","reported_seq":275,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073377+0000","last_change":"2024-09-15T21:01:20.741674+0000","last_active":"2024-09-15T21:02:15.073377+0000","last_peered":"2024-09-15T21:02:15.073377+0000","last_clean":"2024-09-15T21:02:15.073377+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:15.073377+0000","last_undegraded":"2024-09-15T21:02:15.073377+0000","last_fullsized":"2024-09-15T21:02:15.073377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:28.840367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":247,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:16.055967+0000","last_change":"2024-09-15T21:00:07.929456+0000","last_active":"2024-09-15T21:02:16.055967+0000","last_peered":"2024-09-15T21:02:16.055967+0000","last_clean":"2024-09-15T21:02:16.055967+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:16.055967+0000","last_undegraded":"2024-09-15T21:02:16.055967+0000","last_fullsized":"2024-09-15T21:02:16.055967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:18.292639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":246,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411385+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:15.411385+0000","last_peered":"2024-09-15T21:02:15.411385+0000","last_clean":"2024-09-15T21:02:15.411385+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:15.411385+0000","last_undegraded":"2024-09-15T21:02:15.411385+0000","last_fullsized":"2024-09-15T21:02:15.411385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'1","reported_seq":259,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.073404+0000","last_change":"2024-09-15T21:01:48.868446+0000","last_active":"2024-09-15T21:02:15.073404+0000","last_peered":"2024-09-15T21:02:15.073404+0000","last_clean":"2024-09-15T21:02:15.073404+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:15.073404+0000","last_undegraded":"2024-09-15T21:02:15.073404+0000","last_fullsized":"2024-09-15T21:02:15.073404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:01:48.868291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:01:48.868291+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:49.628848+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":369,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-15T21:02:15.411675+0000","last_change":"2024-09-15T21:01:11.181637+0000","last_active":"2024-09-15T21:02:15.411675+0000","last_peered":"2024-09-15T21:02:15.411675+0000","last_clean":"2024-09-15T21:02:15.411675+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:15.411675+0000","last_undegraded":"2024-09-15T21:02:15.411675+0000","last_fullsized":"2024-09-15T21:02:15.411675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:24.330753+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411361,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55580,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316260,"statfs":{"total":96636764160,"available":96579850240,"internally_reserved":0,"allocated":2232320,"data_stored":1583814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411360,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33424,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338416,"statfs":{"total":96636764160,"available":96602537984,"internally_reserved":0,"allocated":2293760,"data_stored":1645254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444064,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32852,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55060,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316780,"statfs":{"total":96636764160,"available":96580382720,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444063,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444065,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444063,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1054905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444064,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":1054886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:20.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:02:21.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:21.145 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":141,"fsid":"07b66685-24ed-4325-b3d4-b0cceb7d7e6d","created":"2024-09-15T20:59:43.863405+0000","modified":"2024-09-15T21:02:19.098427+0000","last_up_change":"2024-09-15T20:59:57.421497+0000","last_in_change":"2024-09-15T21:01:16.623312+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T20:59:55.427902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:00:04.483565+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d1f9d0a1-cdd4-40b0-bec5-e6463b6d4dfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6809","nonce":630056637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6811","nonce":630056637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6815","nonce":630056637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":630056637},{"type":"v1","addr":"172.21.15.27:6813","nonce":630056637}]},"public_addr":"172.21.15.27:6809/630056637","cluster_addr":"172.21.15.27:6811/630056637","heartbeat_back_addr":"172.21.15.27:6815/630056637","heartbeat_front_addr":"172.21.15.27:6813/630056637","state":["exists","up"]},{"osd":1,"uuid":"43960b2a-b250-45e3-b725-484c0751590d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6801","nonce":2797309965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6803","nonce":2797309965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6807","nonce":2797309965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":2797309965},{"type":"v1","addr":"172.21.15.27:6805","nonce":2797309965}]},"public_addr":"172.21.15.27:6801/2797309965","cluster_addr":"172.21.15.27:6803/2797309965","heartbeat_back_addr":"172.21.15.27:6807/2797309965","heartbeat_front_addr":"172.21.15.27:6805/2797309965","state":["exists","up"]},{"osd":2,"uuid":"9653f0df-bf71-4a94-b79a-498f067c8de8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6825","nonce":1130579408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6827","nonce":1130579408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6831","nonce":1130579408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1130579408},{"type":"v1","addr":"172.21.15.27:6829","nonce":1130579408}]},"public_addr":"172.21.15.27:6825/1130579408","cluster_addr":"172.21.15.27:6827/1130579408","heartbeat_back_addr":"172.21.15.27:6831/1130579408","heartbeat_front_addr":"172.21.15.27:6829/1130579408","state":["exists","up"]},{"osd":3,"uuid":"9ac2adf5-f248-4838-9bd5-57d13dd53309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2217694359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6819","nonce":2217694359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6823","nonce":2217694359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2217694359},{"type":"v1","addr":"172.21.15.27:6821","nonce":2217694359}]},"public_addr":"172.21.15.27:6817/2217694359","cluster_addr":"172.21.15.27:6819/2217694359","heartbeat_back_addr":"172.21.15.27:6823/2217694359","heartbeat_front_addr":"172.21.15.27:6821/2217694359","state":["exists","up"]},{"osd":4,"uuid":"72c3acd6-6e64-4853-8823-947f944b96b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6801","nonce":2520277109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6803","nonce":2520277109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6807","nonce":2520277109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":2520277109},{"type":"v1","addr":"172.21.15.66:6805","nonce":2520277109}]},"public_addr":"172.21.15.66:6801/2520277109","cluster_addr":"172.21.15.66:6803/2520277109","heartbeat_back_addr":"172.21.15.66:6807/2520277109","heartbeat_front_addr":"172.21.15.66:6805/2520277109","state":["exists","up"]},{"osd":5,"uuid":"ad64f860-44e7-45a6-bc0d-e85458d80f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6817","nonce":1665320626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6819","nonce":1665320626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6823","nonce":1665320626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":1665320626},{"type":"v1","addr":"172.21.15.66:6821","nonce":1665320626}]},"public_addr":"172.21.15.66:6817/1665320626","cluster_addr":"172.21.15.66:6819/1665320626","heartbeat_back_addr":"172.21.15.66:6823/1665320626","heartbeat_front_addr":"172.21.15.66:6821/1665320626","state":["exists","up"]},{"osd":6,"uuid":"4701a955-3643-4210-800d-7a86ba5155bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6809","nonce":1563776909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6811","nonce":1563776909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6815","nonce":1563776909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1563776909},{"type":"v1","addr":"172.21.15.66:6813","nonce":1563776909}]},"public_addr":"172.21.15.66:6809/1563776909","cluster_addr":"172.21.15.66:6811/1563776909","heartbeat_back_addr":"172.21.15.66:6815/1563776909","heartbeat_front_addr":"172.21.15.66:6813/1563776909","state":["exists","up"]},{"osd":7,"uuid":"abc724fc-25c2-4b2e-b4ba-16b61dbaf89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6825","nonce":3067780245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6827","nonce":3067780245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6831","nonce":3067780245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":3067780245},{"type":"v1","addr":"172.21.15.66:6829","nonce":3067780245}]},"public_addr":"172.21.15.66:6825/3067780245","cluster_addr":"172.21.15.66:6827/3067780245","heartbeat_back_addr":"172.21.15.66:6831/3067780245","heartbeat_front_addr":"172.21.15.66:6829/3067780245","state":["exists","up"]},{"osd":8,"uuid":"b114589d-5020-4c1c-ad76-f07231e58d47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6825","nonce":437424413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6827","nonce":437424413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6831","nonce":437424413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":437424413},{"type":"v1","addr":"172.21.15.92:6829","nonce":437424413}]},"public_addr":"172.21.15.92:6825/437424413","cluster_addr":"172.21.15.92:6827/437424413","heartbeat_back_addr":"172.21.15.92:6831/437424413","heartbeat_front_addr":"172.21.15.92:6829/437424413","state":["exists","up"]},{"osd":9,"uuid":"6e4c938a-4505-41f5-90dc-416c6ceb868a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6801","nonce":591511703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6803","nonce":591511703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6807","nonce":591511703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":591511703},{"type":"v1","addr":"172.21.15.92:6805","nonce":591511703}]},"public_addr":"172.21.15.92:6801/591511703","cluster_addr":"172.21.15.92:6803/591511703","heartbeat_back_addr":"172.21.15.92:6807/591511703","heartbeat_front_addr":"172.21.15.92:6805/591511703","state":["exists","up"]},{"osd":10,"uuid":"fa725c67-48fe-45ac-bcfb-6cf3fddea7fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6809","nonce":2398207120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6811","nonce":2398207120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6815","nonce":2398207120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2398207120},{"type":"v1","addr":"172.21.15.92:6813","nonce":2398207120}]},"public_addr":"172.21.15.92:6809/2398207120","cluster_addr":"172.21.15.92:6811/2398207120","heartbeat_back_addr":"172.21.15.92:6815/2398207120","heartbeat_front_addr":"172.21.15.92:6813/2398207120","state":["exists","up"]},{"osd":11,"uuid":"c6fcefe9-5105-45bf-ac2d-ca8eb186316c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6817","nonce":2517738961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6819","nonce":2517738961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6823","nonce":2517738961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2517738961},{"type":"v1","addr":"172.21.15.92:6821","nonce":2517738961}]},"public_addr":"172.21.15.92:6817/2517738961","cluster_addr":"172.21.15.92:6819/2517738961","heartbeat_back_addr":"172.21.15.92:6823/2517738961","heartbeat_front_addr":"172.21.15.92:6821/2517738961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:03.908437+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:10.056446+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.815936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:20.317862+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:58.417589+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:06.653717+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:07.037212+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:24.672718+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:18.424327+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:17.470601+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:02:17.612257+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:01:15.906985+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":26,"length":1},{"begin":28,"length":3},{"begin":32,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:02:22.164 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:02:22.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:22.329 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:22.329 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:22.329 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:22.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:02:22.645 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:02:22.664 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:02:22.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:22.815 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:22.815 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:22.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:02:23.151 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:02:23.171 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:02:23.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:23.321 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:23.322 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:23.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:02:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:02:23.662 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T21:02:23.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:23.818 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:23.819 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:23.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T21:02:24.138 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T21:02:24.157 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T21:02:24.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:24.305 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:24.305 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:24.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T21:02:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T21:02:24.646 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T21:02:24.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:24.802 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:24.802 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:24.802 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:24.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T21:02:25.120 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T21:02:25.138 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T21:02:25.138 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:25.285 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:25.285 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:25.285 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:25.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T21:02:25.618 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T21:02:25.639 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T21:02:25.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:25.792 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:25.792 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:25.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T21:02:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T21:02:26.136 INFO:tasks.ceph:Scrubbing osd.8 2024-09-15T21:02:26.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:26.294 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:26.294 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:26.294 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:26.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-15T21:02:26.612 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 8 to deep-scrub 2024-09-15T21:02:26.631 INFO:tasks.ceph:Scrubbing osd.9 2024-09-15T21:02:26.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:26.778 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:26.778 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:26.778 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:26.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-15T21:02:27.106 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 9 to deep-scrub 2024-09-15T21:02:27.126 INFO:tasks.ceph:Scrubbing osd.10 2024-09-15T21:02:27.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:27.283 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:27.283 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:27.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-15T21:02:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 10 to deep-scrub 2024-09-15T21:02:27.622 INFO:tasks.ceph:Scrubbing osd.11 2024-09-15T21:02:27.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:02:27.774 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-15T21:02:27.775 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:02:27.775 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-15T21:02:27.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-15T21:02:28.095 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 11 to deep-scrub 2024-09-15T21:02:28.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:28.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:28.424 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:28.441 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-15T21:02:27.418877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":484504,"kb_used_data":21424,"kb_used_omap":18,"kb_used_meta":462829,"kb_avail":1131977576,"statfs":{"total":1159641169920,"available":1159145037824,"internally_reserved":0,"allocated":21938176,"data_stored":14034580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":473937211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.309205"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":265,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:26.519960+0000","last_change":"2024-09-15T21:02:26.519960+0000","last_active":"2024-09-15T21:02:26.519960+0000","last_peered":"2024-09-15T21:02:26.519960+0000","last_clean":"2024-09-15T21:02:26.519960+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:26.519960+0000","last_undegraded":"2024-09-15T21:02:26.519960+0000","last_fullsized":"2024-09-15T21:02:26.519960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:26.519800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:26.519800+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.519800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:47.772479+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:25.582840+0000","last_change":"2024-09-15T21:02:24.923698+0000","last_active":"2024-09-15T21:02:25.582840+0000","last_peered":"2024-09-15T21:02:25.582840+0000","last_clean":"2024-09-15T21:02:25.582840+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:25.582840+0000","last_undegraded":"2024-09-15T21:02:25.582840+0000","last_fullsized":"2024-09-15T21:02:25.582840+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:24.923578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:24.923578+0000","last_clean_scrub_stamp":"2024-09-15T21:02:24.923578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:48.616443+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":270,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:26.947128+0000","last_change":"2024-09-15T21:02:26.947128+0000","last_active":"2024-09-15T21:02:26.947128+0000","last_peered":"2024-09-15T21:02:26.947128+0000","last_clean":"2024-09-15T21:02:26.947128+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:26.947128+0000","last_undegraded":"2024-09-15T21:02:26.947128+0000","last_fullsized":"2024-09-15T21:02:26.947128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:26.946950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:26.946950+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.946950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:28.188829+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:25.482669+0000","last_change":"2024-09-15T21:02:23.861854+0000","last_active":"2024-09-15T21:02:25.482669+0000","last_peered":"2024-09-15T21:02:25.482669+0000","last_clean":"2024-09-15T21:02:25.482669+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:25.482669+0000","last_undegraded":"2024-09-15T21:02:25.482669+0000","last_fullsized":"2024-09-15T21:02:25.482669+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:23.861757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:23.861757+0000","last_clean_scrub_stamp":"2024-09-15T21:02:23.861757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:33.602195+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"142'3","reported_seq":290,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:27.200576+0000","last_change":"2024-09-15T21:02:27.200576+0000","last_active":"2024-09-15T21:02:27.200576+0000","last_peered":"2024-09-15T21:02:27.200576+0000","last_clean":"2024-09-15T21:02:27.200576+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:27.200576+0000","last_undegraded":"2024-09-15T21:02:27.200576+0000","last_fullsized":"2024-09-15T21:02:27.200576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'3","last_scrub_stamp":"2024-09-15T21:02:27.200370+0000","last_deep_scrub":"142'3","last_deep_scrub_stamp":"2024-09-15T21:02:27.200370+0000","last_clean_scrub_stamp":"2024-09-15T21:02:27.200370+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:41.373213+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":259,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-15T21:02:20.326017+0000","last_change":"2024-09-15T21:02:20.326017+0000","last_active":"2024-09-15T21:02:20.326017+0000","last_peered":"2024-09-15T21:02:20.326017+0000","last_clean":"2024-09-15T21:02:20.326017+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:20.326017+0000","last_undegraded":"2024-09-15T21:02:20.326017+0000","last_fullsized":"2024-09-15T21:02:20.326017+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:20.325938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:02:20.325938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:29.830456+0000","scrub_duration":199,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":252,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:25.563170+0000","last_change":"2024-09-15T21:00:08.362462+0000","last_active":"2024-09-15T21:02:25.563170+0000","last_peered":"2024-09-15T21:02:25.563170+0000","last_clean":"2024-09-15T21:02:25.563170+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:25.563170+0000","last_undegraded":"2024-09-15T21:02:25.563170+0000","last_fullsized":"2024-09-15T21:02:25.563170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:00:04.869457+0000","last_clean_scrub_stamp":"2024-09-15T21:00:04.869457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:02:31.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"142'2","reported_seq":274,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-15T21:02:26.173835+0000","last_change":"2024-09-15T21:02:26.173835+0000","last_active":"2024-09-15T21:02:26.173835+0000","last_peered":"2024-09-15T21:02:26.173835+0000","last_clean":"2024-09-15T21:02:26.173835+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:26.173835+0000","last_undegraded":"2024-09-15T21:02:26.173835+0000","last_fullsized":"2024-09-15T21:02:26.173835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'2","last_scrub_stamp":"2024-09-15T21:02:26.173594+0000","last_deep_scrub":"142'2","last_deep_scrub_stamp":"2024-09-15T21:02:26.173594+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.173594+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:47.645402+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"142'94","reported_seq":381,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-15T21:02:27.119423+0000","last_change":"2024-09-15T21:02:26.810710+0000","last_active":"2024-09-15T21:02:27.119423+0000","last_peered":"2024-09-15T21:02:27.119423+0000","last_clean":"2024-09-15T21:02:27.119423+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:27.119423+0000","last_undegraded":"2024-09-15T21:02:27.119423+0000","last_fullsized":"2024-09-15T21:02:27.119423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'92","last_scrub_stamp":"2024-09-15T21:01:11.181514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T20:59:56.411296+0000","last_clean_scrub_stamp":"2024-09-15T21:01:11.181514+0000","objects_scrubbed":2,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1724416,"data_stored":1070452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411362,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2256896,"data_stored":1599380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411361,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1724416,"data_stored":1070452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411361,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411361,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1668578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32884,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444067,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55092,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316748,"statfs":{"total":96636764160,"available":96580349952,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444066,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55092,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316748,"statfs":{"total":96636764160,"available":96580349952,"internally_reserved":0,"allocated":1740800,"data_stored":1078229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444065,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32884,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444067,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444065,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":1078229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444066,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1736704,"data_stored":1078210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:28.445 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:02:20.325938+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=2, tm_sec=20, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=2, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:02:28.446 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:00:04.869457+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=0, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=2, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:02:28.446 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:01:11.181514+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=1, tm_sec=11, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=2, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:02:28.446 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:02:48.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:02:48.765 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-15T21:02:48.765 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-15T21:02:48.783 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-15T21:02:47.424507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":125,"num_write_kb":2152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":489380,"kb_used_data":22144,"kb_used_omap":18,"kb_used_meta":466989,"kb_avail":1131972700,"statfs":{"total":1159641169920,"available":1159140044800,"internally_reserved":0,"allocated":22675456,"data_stored":14500060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":478197051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.865246"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":275,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:43.633745+0000","last_change":"2024-09-15T21:02:26.519960+0000","last_active":"2024-09-15T21:02:43.633745+0000","last_peered":"2024-09-15T21:02:43.633745+0000","last_clean":"2024-09-15T21:02:43.633745+0000","last_became_active":"2024-09-15T21:00:05.991723+0000","last_became_peered":"2024-09-15T21:00:05.991723+0000","last_unstale":"2024-09-15T21:02:43.633745+0000","last_undegraded":"2024-09-15T21:02:43.633745+0000","last_fullsized":"2024-09-15T21:02:43.633745+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:26.519800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:26.519800+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.519800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:47.772479+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00077775100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.115505+0000","last_change":"2024-09-15T21:02:24.923698+0000","last_active":"2024-09-15T21:02:44.115505+0000","last_peered":"2024-09-15T21:02:44.115505+0000","last_clean":"2024-09-15T21:02:44.115505+0000","last_became_active":"2024-09-15T21:01:55.889650+0000","last_became_peered":"2024-09-15T21:01:55.889650+0000","last_unstale":"2024-09-15T21:02:44.115505+0000","last_undegraded":"2024-09-15T21:02:44.115505+0000","last_fullsized":"2024-09-15T21:02:44.115505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:24.923578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:24.923578+0000","last_clean_scrub_stamp":"2024-09-15T21:02:24.923578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:48.616443+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":280,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:43.681078+0000","last_change":"2024-09-15T21:02:26.947128+0000","last_active":"2024-09-15T21:02:43.681078+0000","last_peered":"2024-09-15T21:02:43.681078+0000","last_clean":"2024-09-15T21:02:43.681078+0000","last_became_active":"2024-09-15T21:00:06.891855+0000","last_became_peered":"2024-09-15T21:00:06.891855+0000","last_unstale":"2024-09-15T21:02:43.681078+0000","last_undegraded":"2024-09-15T21:02:43.681078+0000","last_fullsized":"2024-09-15T21:02:43.681078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:26.946950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:26.946950+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.946950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:28.188829+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.00094334499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":81,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.115541+0000","last_change":"2024-09-15T21:02:23.861854+0000","last_active":"2024-09-15T21:02:44.115541+0000","last_peered":"2024-09-15T21:02:44.115541+0000","last_clean":"2024-09-15T21:02:44.115541+0000","last_became_active":"2024-09-15T21:01:55.780864+0000","last_became_peered":"2024-09-15T21:01:55.780864+0000","last_unstale":"2024-09-15T21:02:44.115541+0000","last_undegraded":"2024-09-15T21:02:44.115541+0000","last_fullsized":"2024-09-15T21:02:44.115541+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:23.861757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:23.861757+0000","last_clean_scrub_stamp":"2024-09-15T21:02:23.861757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:33.602195+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"142'3","reported_seq":300,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.115745+0000","last_change":"2024-09-15T21:02:27.200576+0000","last_active":"2024-09-15T21:02:44.115745+0000","last_peered":"2024-09-15T21:02:44.115745+0000","last_clean":"2024-09-15T21:02:44.115745+0000","last_became_active":"2024-09-15T21:01:20.740164+0000","last_became_peered":"2024-09-15T21:01:20.740164+0000","last_unstale":"2024-09-15T21:02:44.115745+0000","last_undegraded":"2024-09-15T21:02:44.115745+0000","last_fullsized":"2024-09-15T21:02:44.115745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'3","last_scrub_stamp":"2024-09-15T21:02:27.200370+0000","last_deep_scrub":"142'3","last_deep_scrub_stamp":"2024-09-15T21:02:27.200370+0000","last_clean_scrub_stamp":"2024-09-15T21:02:27.200370+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:41.373213+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.0026344659999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":277,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-15T21:02:39.097457+0000","last_change":"2024-09-15T21:02:24.433590+0000","last_active":"2024-09-15T21:02:39.097457+0000","last_peered":"2024-09-15T21:02:39.097457+0000","last_clean":"2024-09-15T21:02:39.097457+0000","last_became_active":"2024-09-15T21:00:06.085406+0000","last_became_peered":"2024-09-15T21:00:06.085406+0000","last_unstale":"2024-09-15T21:02:39.097457+0000","last_undegraded":"2024-09-15T21:02:39.097457+0000","last_fullsized":"2024-09-15T21:02:39.097457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:24.433405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:24.433405+0000","last_clean_scrub_stamp":"2024-09-15T21:02:24.433405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:45.462061+0000","scrub_duration":199,"objects_trimmed":0,"snaptrim_duration":0.0010728210000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":270,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.115802+0000","last_change":"2024-09-15T21:02:27.849828+0000","last_active":"2024-09-15T21:02:44.115802+0000","last_peered":"2024-09-15T21:02:44.115802+0000","last_clean":"2024-09-15T21:02:44.115802+0000","last_became_active":"2024-09-15T21:00:06.087543+0000","last_became_peered":"2024-09-15T21:00:06.087543+0000","last_unstale":"2024-09-15T21:02:44.115802+0000","last_undegraded":"2024-09-15T21:02:44.115802+0000","last_fullsized":"2024-09-15T21:02:44.115802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:02:27.849656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:02:27.849656+0000","last_clean_scrub_stamp":"2024-09-15T21:02:27.849656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:49.782473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10084995099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"142'2","reported_seq":284,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.115755+0000","last_change":"2024-09-15T21:02:26.173835+0000","last_active":"2024-09-15T21:02:44.115755+0000","last_peered":"2024-09-15T21:02:44.115755+0000","last_clean":"2024-09-15T21:02:44.115755+0000","last_became_active":"2024-09-15T21:00:06.189289+0000","last_became_peered":"2024-09-15T21:00:06.189289+0000","last_unstale":"2024-09-15T21:02:44.115755+0000","last_undegraded":"2024-09-15T21:02:44.115755+0000","last_fullsized":"2024-09-15T21:02:44.115755+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'2","last_scrub_stamp":"2024-09-15T21:02:26.173594+0000","last_deep_scrub":"142'2","last_deep_scrub_stamp":"2024-09-15T21:02:26.173594+0000","last_clean_scrub_stamp":"2024-09-15T21:02:26.173594+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:47.645402+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0.001428234,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"142'94","reported_seq":396,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-15T21:02:44.215918+0000","last_change":"2024-09-15T21:02:27.220520+0000","last_active":"2024-09-15T21:02:44.215918+0000","last_peered":"2024-09-15T21:02:44.215918+0000","last_clean":"2024-09-15T21:02:44.215918+0000","last_became_active":"2024-09-15T20:59:57.787806+0000","last_became_peered":"2024-09-15T20:59:57.787806+0000","last_unstale":"2024-09-15T21:02:44.215918+0000","last_undegraded":"2024-09-15T21:02:44.215918+0000","last_fullsized":"2024-09-15T21:02:44.215918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'94","last_scrub_stamp":"2024-09-15T21:02:27.220327+0000","last_deep_scrub":"142'94","last_deep_scrub_stamp":"2024-09-15T21:02:27.220327+0000","last_clean_scrub_stamp":"2024-09-15T21:02:27.220327+0000","objects_scrubbed":2,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:03:29.096525+0000","scrub_duration":407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126400,"data_stored":1119296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":19,"seq":81604378662,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1785856,"data_stored":1109242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":18,"seq":77309411366,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55660,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316180,"statfs":{"total":96636764160,"available":96579768320,"internally_reserved":0,"allocated":2318336,"data_stored":1638170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":54655428},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":18,"seq":77309411365,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55140,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316700,"statfs":{"total":96636764160,"available":96580300800,"internally_reserved":0,"allocated":1785856,"data_stored":1109242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":54655424},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":18,"seq":77309411365,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":18,"seq":77309411365,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2392064,"data_stored":1707368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":17,"seq":73014444070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":17,"seq":73014444071,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55152,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316688,"statfs":{"total":96636764160,"available":96580288512,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":17,"seq":73014444070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55156,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316684,"statfs":{"total":96636764160,"available":96580284416,"internally_reserved":0,"allocated":1802240,"data_stored":1117019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":17,"seq":73014444069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":17,"seq":73014444071,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":17,"seq":73014444069,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":1117019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":17,"seq":73014444070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":1117000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:02:48.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:02:49.124 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:02:49.124 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:02:49.124 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:02:49.124 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.220 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:02:49.220 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:02:49.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.308 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:02:49.308 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:02:49.308 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.401 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:02:49.402 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:02:49.402 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.492 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:02:49.492 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T21:02:49.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.579 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T21:02:49.579 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T21:02:49.579 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.657 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T21:02:49.658 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T21:02:49.658 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:49.732 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T21:02:49.733 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T21:02:49.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:55.834 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T21:02:55.835 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-15T21:02:55.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:55.912 INFO:tasks.ceph.osd.8:Stopped 2024-09-15T21:02:55.913 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-15T21:02:55.913 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:55.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-15T21:02:55.994 INFO:tasks.ceph.osd.9:Stopped 2024-09-15T21:02:55.994 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-15T21:02:55.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.060 INFO:tasks.ceph.osd.10:Stopped 2024-09-15T21:02:56.061 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-15T21:02:56.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.123 INFO:tasks.ceph.osd.11:Stopped 2024-09-15T21:02:56.123 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:02:56.124 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:02:56.124 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.154 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:02:56.154 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:02:56.154 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:02:56.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.177 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:02:56.178 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:02:56.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.217 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:02:56.218 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:02:56.218 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:02:56.242 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:02:56.242 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:02:56.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-15T21:02:56.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T21:02:56.263 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:02:56.592 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T21:02:56.592 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:02:56.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T21:02:56.847 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:02:57.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T21:02:57.030 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T21:02:57.197 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T21:02:57.197 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T21:02:57.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T21:02:57.366 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T21:02:57.564 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T21:02:57.565 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T21:02:57.759 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T21:02:57.759 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T21:02:57.945 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T21:02:57.945 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-15T21:02:58.290 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T21:02:58.290 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-15T21:02:58.516 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T21:02:58.516 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-15T21:02:58.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T21:02:58.715 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-15T21:02:58.900 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:02:58.902 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/data/mon.a.tgz 2024-09-15T21:02:58.902 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-15T21:02:58.908 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.vQRk9OvLnO 2024-09-15T21:02:58.908 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vQRk9OvLnO 2024-09-15T21:02:59.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.vQRk9OvLnO 2024-09-15T21:02:59.112 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.vQRk9OvLnO is 366KB 2024-09-15T21:02:59.173 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.vQRk9OvLnO 2024-09-15T21:02:59.179 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/data/mon.b.tgz 2024-09-15T21:02:59.179 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-15T21:02:59.185 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.rGN16RBUoO 2024-09-15T21:02:59.185 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rGN16RBUoO 2024-09-15T21:02:59.352 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.rGN16RBUoO 2024-09-15T21:02:59.416 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.rGN16RBUoO is 501KB 2024-09-15T21:02:59.477 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.rGN16RBUoO 2024-09-15T21:02:59.483 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/data/mon.c.tgz 2024-09-15T21:02:59.483 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-09-15T21:02:59.489 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.2VItY0TdU7 2024-09-15T21:02:59.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2VItY0TdU7 2024-09-15T21:02:59.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.2VItY0TdU7 2024-09-15T21:02:59.720 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.2VItY0TdU7 is 501KB 2024-09-15T21:02:59.787 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.2VItY0TdU7 2024-09-15T21:02:59.792 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:02:59.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:02:59.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:02:59.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:02:59.856 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:02:59.862 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:02:59.899 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:02:59.899 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:02:59.901 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi027/crash 2024-09-15T21:02:59.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:02:59.919 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi066/crash 2024-09-15T21:02:59.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:02:59.935 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi092/crash 2024-09-15T21:02:59.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:02:59.955 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:02:59.955 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:02:59.960 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:02:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38772.log 2024-09-15T21:02:59.980 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-09-15T21:02:59.980 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38772.log.gz 2024-09-15T21:02:59.980 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31200.log 2024-09-15T21:02:59.981 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-09-15T21:02:59.982 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35339.log 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31200.log: /var/log/ceph/ceph-client.admin.35339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35339.log.gz 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31507.log 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31200.log.gz 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38434.log 2024-09-15T21:02:59.985 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31507.log.gz 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38616.log 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38434.log.gz 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38616.log.gz 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-15T21:02:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-15T21:02:59.989 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-09-15T21:02:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31052.log 2024-09-15T21:02:59.989 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-15T21:02:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35741.log 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31052.log.gz 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34737.log 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35741.log.gz 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36385.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz -5 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39240.log 2024-09-15T21:02:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34737.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39032.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39240.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36848.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37817.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39032.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34064.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36848.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37817.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37817.log.gz 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32325.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34064.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.34064.log.gz --verbose 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.32659.log 2024-09-15T21:02:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32623.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32325.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32659.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31195.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34486.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32623.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33053.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31195.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34486.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36328.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33053.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31179.log 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36328.log.gz 2024-09-15T21:03:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33392.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.31179.log: /var/log/ceph/ceph-client.admin.34690.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31179.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33392.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31191.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34302.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34690.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31191.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39110.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34302.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35088.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.33204.log 2024-09-15T21:03:00.001 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39110.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33204.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34245.log 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log/var/log/ceph/ceph-client.admin.33939.log: 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34245.log.gz 2024-09-15T21:03:00.002 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33143.log 2024-09-15T21:03:00.003 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38408.log 2024-09-15T21:03:00.003 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2024-09-15T21:03:00.003 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-09-15T21:03:00.003 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33143.log.gz 2024-09-15T21:03:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T21:03:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mon.b.log: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:03:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.4.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:03:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38408.log: /var/log/ceph/ceph-client.admin.37446.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38408.log.gz 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36187.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37446.log.gz 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37238.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36034.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz /var/log/ceph/ceph-client.admin.37818.log 2024-09-15T21:03:00.029 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37238.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31816.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37264.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37818.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31816.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33715.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37264.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37082.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33715.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.38954.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37134.log 2024-09-15T21:03:00.030 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37082.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36411.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38954.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37134.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31173.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36411.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32278.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37498.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31173.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33634.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32278.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37498.log.gz 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-09-15T21:03:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38184.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33634.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35164.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38824.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38184.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32533.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35164.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38824.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32144.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35587.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32533.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32144.log.gz 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38876.log 2024-09-15T21:03:00.032 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38564.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35587.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38179.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38876.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38564.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35388.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38179.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38928.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35388.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38590.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38928.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32464.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32563.log 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38590.log.gz 2024-09-15T21:03:00.033 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32464.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34767.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31884.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32563.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33249.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34767.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35507.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31884.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33249.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31784.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5/var/log/ceph/ceph-client.admin.35507.log: --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35507.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32655.log 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31784.log.gz 2024-09-15T21:03:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32655.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38902.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38902.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32901.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39188.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32901.log.gz 2024-09-15T21:03:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39188.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32118.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39058.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39214.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32118.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39058.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36567.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39214.log.gz 2024-09-15T21:03:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36567.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34462.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31476.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34462.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35218.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32770.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31476.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35218.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31502.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32770.log.gz 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31502.log.gzgzip 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35115.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-15T21:03:00.037 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31780.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35115.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35014.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31780.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35014.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31473.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38798.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35769.log 2024-09-15T21:03:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31473.log.gz 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38798.log.gz 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38233.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35769.log.gz 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36138.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38233.log.gz/var/log/ceph/ceph-client.admin.36138.log: 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36138.log.gz 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gz 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31864.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31190.log 2024-09-15T21:03:00.039 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-09-15T21:03:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26559.log 2024-09-15T21:03:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-15T21:03:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26535.log 2024-09-15T21:03:00.053 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26559.log.gz 2024-09-15T21:03:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26439.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26535.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26535.log.gz 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26439.log.gz 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26343.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26343.log.gz 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26487.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26511.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26487.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26487.log.gz 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26367.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26511.log.gz 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26319.log 2024-09-15T21:03:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26367.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26367.log.gz 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26319.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26319.log.gz 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26415.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.9.log: 85.9% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26391.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26583.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26415.log.gz 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26391.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26391.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26463.log 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26583.log.gz/var/log/ceph/ceph-osd.10.log: 2024-09-15T21:03:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.26463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26463.log.gz 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31864.log.gz 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36095.log 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34163.log 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31190.log.gz/var/log/ceph/ceph-client.admin.36095.log: 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.36095.log.gz --verbose 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.33541.log 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34163.log.gz 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33787.log 2024-09-15T21:03:00.071 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38512.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35311.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33787.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38512.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33864.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31485.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35311.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38668.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33864.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30928.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31485.log.gz/var/log/ceph/ceph-client.admin.38668.log: 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38668.log.gz 2024-09-15T21:03:00.072 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30928.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37212.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37212.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31193.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37495.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31193.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35888.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37495.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35888.log.gz 2024-09-15T21:03:00.073 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37819.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36765.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37819.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36765.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32503.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31181.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32503.log.gz 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35663.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31506.log 2024-09-15T21:03:00.074 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31181.log.gz 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35663.log.gz 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35912.log 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32196.log 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31506.log.gz 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35912.log.gz 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38642.log 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32947.log 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32196.log.gz 2024-09-15T21:03:00.075 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38642.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31007.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.32947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.32947.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31007.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31512.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31512.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35979.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34109.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35979.log.gz 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33666.log 2024-09-15T21:03:00.076 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34109.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34109.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33666.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39136.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31508.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39136.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31508.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35049.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31130.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35049.log.gz 2024-09-15T21:03:00.077 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31130.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31078.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35502.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31078.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26363.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35502.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31844.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.26363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26363.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33964.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34891.log 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31844.log: /var/log/ceph/ceph-client.admin.33964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31844.log.gz 2024-09-15T21:03:00.078 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33964.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35638.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34916.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34891.log: /var/log/ceph/ceph-client.admin.35638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34891.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35638.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33446.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34916.log: /var/log/ceph/ceph-client.admin.37186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34916.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36536.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33446.log: /var/log/ceph/ceph-client.admin.36536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33446.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36536.log.gz 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31198.log 2024-09-15T21:03:00.079 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39162.log: /var/log/ceph/ceph-client.admin.37494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32980.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38980.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31198.log: /var/log/ceph/ceph-client.admin.32980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31198.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32980.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37290.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33330.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38980.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37290.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38460.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33330.log.gz 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21415.log 2024-09-15T21:03:00.080 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38460.log: /var/log/ceph/ceph-client.admin.37500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38460.log.gz 2024-09-15T21:03:00.081 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-09-15T21:03:00.081 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37791.log 2024-09-15T21:03:00.081 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35413.log 2024-09-15T21:03:00.082 INFO:teuthology.orchestra.run.smithi066.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.21415.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21415.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37791.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31504.log 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37791.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35413.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38850.log 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32853.log 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31504.log: /var/log/ceph/ceph-client.admin.38850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31504.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38850.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31867.log 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32326.log 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32853.log.gz 2024-09-15T21:03:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31867.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32326.log: /var/log/ceph/ceph.audit.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32326.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36900.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32320.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36900.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gzgzip 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32320.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33025.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33474.log 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-09-15T21:03:00.096 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33025.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31474.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33025.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33474.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31104.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33117.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31474.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31104.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38538.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31104.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33117.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31788.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34413.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38538.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31788.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36739.log 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31788.log.gz 2024-09-15T21:03:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34413.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34567.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36739.log: /var/log/ceph/ceph-client.admin.33890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36739.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34621.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33367.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34567.log: /var/log/ceph/ceph-client.admin.34621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34567.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34621.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37160.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31484.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33367.log.gz 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37160.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30976.log 2024-09-15T21:03:00.098 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31484.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36874.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31174.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30976.log: /var/log/ceph/ceph-client.admin.36874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30976.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36874.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35808.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31174.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35808.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39264.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35808.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32092.log 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz 2024-09-15T21:03:00.099 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32709.log 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32092.log.gz 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36713.log 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33284.log 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32709.log: /var/log/ceph/ceph-client.admin.36713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32709.log.gz 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36713.log.gz 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37513.log 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38107.log 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33284.log: /var/log/ceph/ceph-client.admin.37513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33284.log.gz 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37513.log.gz 2024-09-15T21:03:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34514.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31815.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38107.log.gz 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34514.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36512.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31815.log.gz 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31883.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36512.log: /var/log/ceph/ceph-client.admin.37783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36512.log.gz 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-09-15T21:03:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37813.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31883.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37813.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34763.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38154.log: /var/log/ceph/ceph-client.admin.34763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34763.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31843.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32222.log: /var/log/ceph/ceph-client.admin.38177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32222.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32740.log 2024-09-15T21:03:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36054.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31843.log: /var/log/ceph/ceph-client.admin.32740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31843.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32740.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34865.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36054.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34865.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34339.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34865.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32593.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39084.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34339.log: /var/log/ceph/ceph-client.admin.32593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34339.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32593.log.gz 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-09-15T21:03:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-09-15T21:03:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39084.log.gz 2024-09-15T21:03:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-09-15T21:03:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33591.log 2024-09-15T21:03:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-09-15T21:03:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38720.log 2024-09-15T21:03:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31493.log 2024-09-15T21:03:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33591.log: /var/log/ceph/ceph-client.admin.38720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33591.log.gz 2024-09-15T21:03:00.116 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38720.log.gz 2024-09-15T21:03:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36213.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31493.log: /var/log/ceph/ceph-client.admin.36213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31493.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36213.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26173.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35803.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39006.log: /var/log/ceph/ceph-client.admin.26173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26173.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38746.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35803.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38746.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32318.log 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38746.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-09-15T21:03:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32170.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36638.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32318.log: /var/log/ceph/ceph-client.admin.32170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32318.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32170.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31196.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36262.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36638.log: /var/log/ceph/ceph-client.admin.31196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36638.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31196.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34187.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38108.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36262.log: /var/log/ceph/ceph-client.admin.34187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34187.log.gz 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38486.log 2024-09-15T21:03:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31869.log 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38108.log: /var/log/ceph/ceph-client.admin.38486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38108.log.gz 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38486.log.gz 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32798.log 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31869.log: /var/log/ceph/ceph-client.admin.32798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31869.log.gz 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32798.log.gz 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31032.log 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26257.log 2024-09-15T21:03:00.121 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31032.log: /var/log/ceph/ceph-client.admin.38135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-09-15T21:03:00.122 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31032.log.gz 2024-09-15T21:03:00.122 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38153.log 2024-09-15T21:03:00.122 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.26257.log: 0.0%/var/log/ceph/ceph-client.admin.38153.log: -- replaced with /var/log/ceph/ceph-client.admin.26257.log.gz 2024-09-15T21:03:00.122 INFO:teuthology.orchestra.run.smithi027.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38153.log.gz 2024-09-15T21:03:00.164 INFO:teuthology.orchestra.run.smithi092.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:03:00.171 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:03:00.367 INFO:teuthology.orchestra.run.smithi027.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:03:00.673 INFO:teuthology.orchestra.run.smithi027.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:03:00.736 INFO:teuthology.orchestra.run.smithi092.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-15T21:03:00.796 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-15T21:03:00.826 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T21:03:00.846 INFO:teuthology.orchestra.run.smithi027.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:03:00.901 INFO:teuthology.orchestra.run.smithi066.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T21:03:00.931 INFO:teuthology.orchestra.run.smithi066.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T21:03:00.950 INFO:teuthology.orchestra.run.smithi027.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T21:03:00.966 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-15T21:03:01.042 INFO:teuthology.orchestra.run.smithi027.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:03:01.044 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-15T21:03:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m1.081s 2024-09-15T21:03:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m3.897s 2024-09-15T21:03:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.196s 2024-09-15T21:03:01.229 INFO:teuthology.orchestra.run.smithi092.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-15T21:03:01.557 INFO:teuthology.orchestra.run.smithi092.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-15T21:03:01.559 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-15T21:03:01.559 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m1.551s 2024-09-15T21:03:01.559 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m4.184s 2024-09-15T21:03:01.559 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.193s 2024-09-15T21:03:01.702 INFO:teuthology.orchestra.run.smithi066.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T21:03:01.703 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-15T21:03:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m1.721s 2024-09-15T21:03:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m4.476s 2024-09-15T21:03:01.704 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.157s 2024-09-15T21:03:01.705 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:03:01.705 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi027/log 2024-09-15T21:03:01.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:03:02.378 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi066/log 2024-09-15T21:03:02.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:03:02.931 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi092/log 2024-09-15T21:03:02.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:03:03.476 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:03:03.486 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:03:03.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:03:03.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:03:03.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:03:03.587 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:03:03.596 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:03:03.596 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:03:03.599 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:03:03.601 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:============================================================================== 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:+hv01.front.sepi 67.205.162.81 3 u 46 64 77 0.094 -1.841 2.274 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:*hv02.front.sepi 63.231.80.2 3 u 46 64 77 0.147 -1.024 2.229 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:+hv03.front.sepi 74.6.168.72 3 u 42 64 77 0.135 +0.538 2.102 2024-09-15T21:03:03.610 INFO:teuthology.orchestra.run.smithi027.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================== 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout:+hv01.front.sepi 67.205.162.81 3 u 43 64 77 0.101 -1.754 0.332 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout:*hv02.front.sepi 63.231.80.2 3 u 43 64 77 0.091 -1.212 0.247 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 77 0.094 +0.322 0.189 2024-09-15T21:03:03.612 INFO:teuthology.orchestra.run.smithi066.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout:*hv01.front.sepi 67.205.162.81 3 u 43 64 77 0.123 -2.023 1.318 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout:+hv02.front.sepi 63.231.80.2 3 u 42 64 77 0.209 -1.241 1.307 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 77 0.142 +0.292 1.361 2024-09-15T21:03:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:03:03.614 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:03:03.633 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:03:03.634 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:03:03.667 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:03:03.700 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:03:03.734 INFO:teuthology.task.internal:Duration was 846.833500 seconds 2024-09-15T21:03:03.734 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:03:03.767 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:03:03.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:03:03.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:03:03.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:03:03.810 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:03:03.810 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-15T21:03:03.810 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:03:03.864 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-09-15T21:03:03.865 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:03:03.875 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-15T21:03:03.876 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:03:03.886 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:03:03.886 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:03:03.908 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:03:03.920 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:03:03.956 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:03:03.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:03:03.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:03:03.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:03:04.062 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:03:04.072 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:03:04.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:03:04.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:03:04.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:03:04.109 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:03:04.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.128 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.137 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-15T21:03:04.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.149 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-09-15T21:03:04.165 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-15T21:03:04.190 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:04.203 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:04.223 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:03:04.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:04.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:03:04.250 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:03:04.251 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi027 2024-09-15T21:03:04.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:03:04.274 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi066 2024-09-15T21:03:04.275 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:03:04.298 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905637/remote/smithi092 2024-09-15T21:03:04.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:03:04.319 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:03:04.319 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:03:04.322 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:03:04.336 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:03:04.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:03:04.375 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:03:04.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:03:04.401 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:03:04.401 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:03:04.403 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:03:04.405 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:03:04.407 INFO:teuthology.orchestra.run.smithi027.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:03 /home/ubuntu/cephtest 2024-09-15T21:03:04.409 INFO:teuthology.orchestra.run.smithi066.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:03 /home/ubuntu/cephtest 2024-09-15T21:03:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:03 /home/ubuntu/cephtest 2024-09-15T21:03:04.412 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:03:04.435 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:03:04.506 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-bitmap} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} duration: 846.8335003852844 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:03:04.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:03:04.592 INFO:teuthology.run:pass